Show HN: Microsoft releases Flint, a visualization language for AI agents(microsoft.github.io)
40 pointsby chenglong-hnan hour ago10 comments
  • dvt13 minutes ago
    This is pretty crazy, literally built something almost exactly like this for a project I'm working on (a local-first AI agent that does work on folders while you sleep). Basically going from JSON "Lego blocks" to full reports (including charting, though a subset of what Flint offers). And with post-generation validation and retry steps.

    Functions extremely well and the result is a very clear (and consitent) human-readable "output layer." Cool idea, fun to see people converging on similar concepts in the space.

    • chenglong-hn9 minutes ago
      That's awesome!

      I find that besides training better models, designing new language for agents is also a super viable paths to improve their performance!

  • FailMorean hour ago
    The charts are very nice, and I think the visualisation layer for LLMs is a very interesting problem.

    I’ve been building https://smalldocs.org for this exact reason. It’s an office suite for AI agents - but my main use case is giving a cli based LLM the canvas to express itself - charts, mermaid diagrams, etc. I’ve extended it a bit further to be a format for all types of work so the agent can embed slides and spreadsheets in a document.

    Sample document: https://smalldocs.org/blogs/what-is-a-smalldoc

    Source: https://github.com/espressoplease/smalldocs

    • giancarlostoroan hour ago
      > mermaid diagrams

      I'm terrible at diagrams, so I gave GPT very generic descriptions of one of our project, to convert in to that mermaid style, then for Lucid I pasted it in there, and had a visualization of what I needed. Worked out nicely.

  • theKan hour ago
    > requiring them to explicitly make visual decisions that are supposed to be handled by a good compiler

    Isnt graphviz there for the same reason?

    Edit: I see it is using JSON as the declaration language, I am OK with llms being "good at json" but a syntax also consumable by humans it is not!

    • chenglong-hn34 minutes ago
      In fact, Json as a common language for human in visualization has been around for a while! The benefit of declarative grammar is that users can effective manipulate specs through UI (drag and drop, clicks).

      Btw, Flint is intentionally designed to allow agent skip low-level params like scale, axe, zero, step size etc (which are extremely crucial for "GOOD-looking") and they are dynamically optimized by the compiler. So AI agents can have a easier time.

      • theK15 minutes ago
        > Json as a common language for human in visualization has been around for a while

        Plant, Mermaid, Graphviz are all declarative textual representations designed for human authoring, JSON is made for tools. Its not a criticism just a statement that if interop across agent and human was intended this is not the simplest option.

  • kveykva17 minutes ago
    Is there a specific explanation about how this is better or different than vega itself? https://vega.github.io/vega/docs/specification/

    My understanding is that Vega was already an expressive DSL for visualizations and its probably already well spread through LLM training data.

    • chenglong-hn9 minutes ago
      Vega was a high-level language in the past for human, but now they can be a bit too low-level for AI agents! AI agents have to write a lot of low-level params just to make charts looking good, and the result is that programs are hard to write reliably for AI agents.

      Flint is a higher-level abstraction, with simpler much shorter spec, and the compiler derives low-level decisions so that charts are looking good.

      So: flint lets agent write short program that achieving good looking charts that had to be done with lengthy program in the past.

  • neomantra30 minutes ago
    This is cool to see from a research team. A few weeks ago I was exploring a similar idea with ntcharts, where a user or LLM can specify a chart in a Golang or JSON object...

    and then that spec would be rendered either to a Bubble TUI via NTCharts or to HTML/SVG via ECharts. That Echarts HTML could be naturally served by a Golang http service.

    But Flint goes much deeper with semantic layers and settings optimizations. Perhaps a NTChart, or whatever terminal chart, could be a rendering target? I'll add it to the list to explore...

    https://github.com/NimbleMarkets/ntcharts/blob/spec/spec/REA...

    • chenglong-hn13 minutes ago
      This is fun! We started thinking it would just be an engineering task in the beginning, but doing a solid intermediate language turned out to be a research project (the paper will be out soon).

      Also, I find NTChart very fun, maybe we should add NT chart to the list of compilation backend for Flint so it works in the library. Putting a reminder here: https://github.com/microsoft/flint-chart/issues/45

  • santiagobasulto20 minutes ago
    Forget AI agents, this DSL is better even for humans. Cool project!
  • animal_spiritsan hour ago
    It compiles into Echarts, but echarts already has a JSON co figuration spec
    • chenglong-hn38 minutes ago
      It's more like a simple high-level spec to make it easier. The idea is that you don't have to fill position / axes details just to make the chart work. The compiler has a bit of magic of using semantic types to optimize what parameters will be set in ECharts.

      In some composite chart examples, the good-looking echart spec is like 5x longer than the simple Flint one!

  • nttylocka minute ago
    [flagged]