This guy – whose RSS feed I subscribe to – makes a living [0] by being anti-AI.

That's fine. Do what you do. But don't read this article as any sort of science. It's massively opinionated rage-bait.

[0]: https://www.patreon.com/davidgerard

On my last project, it kept trying to use the system python instead of the project's virtual environment. It also kept using the wrong build tool. Both things wasted considerable tokens because the agent got sidetracked trying to understand why it could not run the tests - and that repeated on each new session.

A simple instruction in AGENTS.md fixed that.

It mentions that a chatbot generated AGENTS.md does nothing, which makes sense.

I added one when it kept making the same mistake and using things from the wrong library version making compiler errors, adding in the common mistakes pre emptively.

“Don’t start off by git diff, git status, or git show” saved me quite a few tokens.

Just because it doesn't materially impact task success rates does not mean it's not useful. I use that file to give my agent information about the environment (operating system, architecture, command-line utilities I have installed), as well as how to do certain things (such as using `uv` for running Python when needed). I want my agent to work how I do, so I also tell it things like my preferred version control system (jj), my preferred implementation languages for things like shell scripts (zsh), and other things like that. I care more about how the work is done than the final result. It still slips up sometimes, but on the whole I think it works alright. It's not like it fixes tasks that wouldn't be completed at all, but it does help my satisfaction with how they were completed, as well as with the final result. Otherwise, I'd have to do a lot more manual cleanup.

Like all LLM related issues, it doesn't until it does. Or does until it doesn't.

Either way, you get no visibility or predictability.

Good luck.