I expected types to give the agent extra information, so it would be faster, use fewer tokens and be right more often. But it did nothing of that, no matter the types. Moreover, with more types it sometimes used more tokens, because there’s, well, more code.
The one thing that did change anything was giving the agent a type checker. It ran mypy on nearly every change, and no type errors got through. So give your AI a feedback tool - like type checker, to keep it in check!
Caveats: this is a case study, not a benchmark. The main codebase was written with Claude, one of the agents tested, and I had only 5–20 tasks per comparison, because tokens aren't free. At the end of the day I used 170 million of them, so you don't have to.