3 pointsby zachdotai9 hours ago2 comments
  • zachdotai9 hours ago
    AI agents break in ways traditional software doesn't. Logic bugs, reasoning failures, edge cases that manual testing and static benchmarks don't fully explore.

    Nyx is an autonomous adversarial harness that probes your agents for vulnerabilities. Since agents are non-deterministic, it can be hard to find the gaps by just reading code. So it interacts with your AI agents in blackbox mode to surface issues across security, logic, and alignment at scale, before they reach users. It's also massively parallel by default

    Instead of spending time writing static evals for the key failure modes of your AI agents, point Nyx at any system and it autonomously discovers failure modes that matter. It can typically find issues in under 10 minutes that manual audits take hours to surface.

    This is early work and we know the methodology is still going to evolve. We would love nothing more than feedback from the community as we iterate on this.

  • natloz8 hours ago
    What happens if you point Nyx at itself, who breaks first!
    • zachdotai8 hours ago
      We're doing that internally to continuously improve our own agent and make it robust against adversarial attacks itself. We will release some insights about self-improvement soon!
      • natloz6 hours ago
        Cool, sounds like an interesting experiment!