37 pointsby bnchrch6 hours ago4 comments
  • danielvlopes26 hours ago
    Hey HN! I'm Daniel, cofounder of GrowthX and Ben's colleague (who posted it). We have about 20 engineers building AI agents and workflows for companies like Lovable, Webflow, Airbyte. Output is the framework we extracted from that work. It runs our AI infrastructure and we open-sourced it.

    We kept hitting the same problems: writing and iterating on prompts at scale, orchestrating API calls that fail unpredictably, tracking costs, testing non-deterministic code, building datasets from production data, organizing repos so coding agents perform well. And every piece of tooling was a different SaaS product that didn't talk to the others.

    We built Output around three ideas:

    1. Make it easy for devs and coding agents to create and modify workflows in one or a few shots.

    Filesystem first. Everything your agent needs lives in self-contained folders, full context visible without hunting. TypeScript and Zod provide the first validation layer for whether your workflow is correct.

    2. One framework, minimal tooling sprawl.

    We got tired of scattering data across SaaS products that don't talk to each other. Prompt files, evals, tracing, cost tracking, credentials all live in one place.

    Your data stays on your infrastructure. Under the hood, we built on Temporal for orchestration. It's a hard problem and we weren't going to reinvent the wheel they've perfected. Open source and self-hostable, or Temporal Cloud. We wrapped it so you don't need to learn Temporal upfront, but the full power is there underneath.

    3. A flat learning curve.

    Our team is web engineers at different levels. We didn't want anyone to learn Python, five different tools, or the nuances of workflow idempotency before they could ship. We baked in conventions: same folder structure, file names, patterns across every workflow. Advanced features like Temporal primitives, evals, LLM-as-a-judge stay out of the way until you reach for them.

    We've been building production workflows this way for over a year.

    We extracted it, cleaned it up, and wanted to put it in front of people who'd push on it.

    Docs and a video building a HN AI digest newsletter from scratch: https://output.ai

    Happy to answer questions.

  • stevenkoze025 hours ago
    The credential management piece is smart most frameworks just do .env files and hope for the best. Curious about one thing: when workflows call external tools or ingest tool descriptions from MCP servers, are you doing any sanitization on the input before it hits the model's context? We've been researching invisible Unicode in tool descriptions codepoints that render as nothing but get tokenized normally. GPT-5.4 follows hidden instructions encoded this way 100% of the time in our testing. At 500+ production agents that's a real attack surface if any of them consume external tool definitions.
    • globalchatads5 hours ago
      The Unicode injection is a real vector, but I keep running into a problem one step before that: how do you even know which MCP servers to trust with tool definitions?

      The official MCP Registry is basically a flat list. No verification metadata, no attestation chain. If someone gets a malicious server listed there, Unicode tricks in tool descriptions are almost beside the point. Your agents are already pulling definitions from an unvetted source.

      I have been tracking the IETF drafts that try to solve agent discovery and registration. There are about 11 competing ones (ARDP, AID, AINS, agents.txt, etc). Six expired or are expiring this month, no renewals filed. The ones still alive do not include any mechanism for cryptographic verification of tool descriptions.

      At 500 agents, the question stops being "is this tool description clean" and becomes "should my agent be talking to this server at all." The sanitization work matters, but it is downstream of a trust problem that is currently wide open.

      • stevenkoze022 hours ago
        The trust problem is real and it's upstream of everything we're scanning for. If the server itself is untrusted then sanitizing its tool descriptions is defense-in-depth at best.

        The 11 competing IETF drafts is a good data point. We looked at the same fragmentation from the A2A side in our paper on Google's Agent2Agent protocol. Six structural gaps in the v1.0 spec, including no authorization model at all. Each agent is an authorization island.

        Our take is that protocol-level trust verification and tool-level sanitization are both necessary. Neither is sufficient alone. The trust layer tells you whether to connect. The sanitization layer tells you whether the content is clean after you've decided to connect. Attackers compromise trusted sources too.

    • bnchrch5 hours ago
      Hey! Ben here (one of the engineers who built this).

      This is a reason why we made our http framework (@outputai/http) a first class citizen for the greater framework and our claude code plugins.

      As you pointed out at this moment in time theres a Cambrian explosion both in new tools/libraries and the willingness to use them, which poses a systemic security threat when combined with how LLMs function.

      So while you're free to use any third party tool or library you want with Output. We encourage you to roll your own as often as possible both for the security/control it gives you. But also for the vertical integration it provides (debugging, cost tracking, evals etc...)

    • marcosmarxm5 hours ago
      Do you mind sharing any content from your team's research? I've recently gotten interested in agent/llm attacks and how to protect against them.
      • stevenkoze022 hours ago
        We've published 5 papers covering different attack surfaces in the AI agent ecosystem: https://agentsid.dev/research

        The most relevant to what you're asking about is "Invisible Ink" (invisible Unicode smuggling in MCP tool descriptions, GPT-5.4 follows hidden instructions 100% of the time) and "Weaponized by Design" (census-scale analysis of 15,982 MCP servers, 137,070 findings). Both have full methodology and reproducibility details.

        The scanner is open source if you want to test against your own stack: npx @agentsid/scanner

  • dp056 hours ago
    Looks great. Sharing with my team
  • johnwhitman2 hours ago
    [dead]