I really don't get the tendency here from Anthropic to describe what's most likely churn? 1000 "PRs" a day on a TUI to wrap a while loop? I'm not going to be a reductionist here so I'm going to cut it off there, I'm positive that the complexity of handling multiple terminal emulators is something that needs some handling (even if doing their own rendering engine was a self made problem). Yes I've vibe coded my own harness from scratch without writing any of it, I still know how it works.
I'm also not sure that they addressed point 2 in any actual capacity? They say "the system verifies itself" which is cool and is something that automated testing has done for decades, but they eschew the mental model of how the system works as a whole and prefer instead to have the LLM focus and summarize a slice of the system when they need to know about it or ask for it. But.... That's not knowledge? That's just search a la google? And thus seems to be subject to the same problems where the quality of your search dictates the quality of your knowledge that you surface at any given time?
I know some people will groan hearing the phrase again, but: "Reverse centaur", you're the machine's tool, rather than the reverse.
> Most of the path is automated; a person's attention goes where the risk is
> The eval grows with every incident, and every change to the reviewer is scored against all of it.
Opus 5.5 is obviously stellar but come on what are you doing lol
And now this. Suffice to say, my efforts did not bear fruit. I feel more isolated than ever before.
I don't think we've gotten to that level of reliable review with code, but I don't see any reason we can't continue moving in that direction.
For code, I think things go in the opposite direction: the system outputs artifacts that are all different among themselves (even more so now that we no longer distribute software via physical copies. Well, bits are always physical, but I mean we no longer ship floppies/CDs/DVDs).
That said, at least in my experience, code review has never been a direct part of factory-line-type QA. That QA is typically done by treating software as a black box and just verifying that it complies with a spec (often times based on a very loose definition of "spec" ...), and this is something that works equally well regardless of whether a human or a machine wrote the software. The type of QA that is done (among other things) by code reviews is checking whether the software's design is such that it will be economically feasible to maintain and extend it, for example.
The one part the code reviews takes a part of (but is in no way error-free) and that you just can't do with black-box testing (or it is much harder to do) is to look for things like backdoors. As our industry moves more towards "agents write software and tests validate that it works," I think a lot about "Reflections on Trusting Trust" and the new and wide attack surface added by people no longer reading the code shipped on their behalf.
Of course Anthropic would love you to think you are heading towards that future so we have to bear in mind their livelihood depends on LLMs being the future of work and 30k changes a day being normal.
But why would we want 30k changes a day in a given product, what possible justification could there be?
First, I think we overestimate how different code is from other code. We even joke about CRUD apps and how cookie-cutter they are. A lot of changes in those apps are: add some new features to a data access layer (SQL, ORM, whatever), add new routes, add new UI surface, etc. There is an entire industry around "Design Patterns" and people live by the DRY axioms. Things like Ruby on Rails exists because rebuilding these systems over and over is wasteful. I feel we are just moving further down that same path.
Second, your question of "why would we want" is totally backwards. Progress doesn't follow your desires. Progress doesn't require justification. Sometimes we adapt the environment to our needs, other times we adapt to a changing environment. Which case we are currently in is still undecided and you are free to fight for the future you prefer. But demanding an industry justify itself before it is allowed to change fails in the face of evolution. If it works, we will justify it in hindsight.
> None required once automated review and security checks are green
How anybody can let an LLM write docs and not even read them is pure insanity.
Authors should really try and view this article from the perspective of someone who has to think about cost to use the models.
"According to the report, adjusted operating income excludes stock-based compensation, which is not unusual for technology companies. However, the report also quotes its gross margins - above 80% - exclude revenue shared with distribution partners, as well as the cost of training its models."
From https://www.morningstar.com/news/marketwatch/2026091414/the-...
Antropic's revenue is up [1], [2], [3] and if you calculate adjusted operating income (by excluding most expenses like training costs) then by that measure it is profitable. However, if you factor in all the numbers it is operating at a loss.
[1] https://finance.yahoo.com/technology/ai/articles/anthropic-v...
[2] https://fortune.com/2026/09/29/anthropic-leaked-ipo-prospect...
[3] https://www.cnbc.com/2026/10/15/anthropics-third-quarter-rev...
And what they're reporting is adjusted even further than EBITDA
Its like telling other ppl about your dreams. Not interesting.
Honestly I’m not worried about code anymore in my codebase I’ve not had bug problems for a while and incidents get fixed automatically.
I monitor new features with an agent who checks recordings.
Most actually good features need still 20-30 iterations to design the userflow to actually be interactive, but then there is also another challenge to translate usage patterns into features and how to present functionality with the least amount of friction as possible.
How to efficiently run monitors without wasting your token spends. A best 'monitor' best practice would be nice.
Still this article was very helpful and gave me a lot of new ideas.
I hope this is only for low-risk false positives. Dismissing a critical finding as a false positive should still involve AppSec.
You could improve revert rates while simultaneously introducing more security vulnerabilities.
And the linked "claude-code-security-review" explicitly states in the README
> This action is not hardened against prompt injection attacks
That's a significant limitation for an AI security review control, no?
> The goal hasn't changed: less human time per change, and no more reverts than before.
Is this Goodhart's law in action? Less reverts and less incidents was measured and became the target. Is this then still a good measure? I guess Revenue is a more direct measure, but we don't have access to that one.
New features have actually been remarkable. For me they are almost always in one of two categories: things I've always wanted (but never actually bothered communicating to the devs), and things I didn't know I needed but did. The product team is clearly very tuned in.
All in all, I'd say their approach is working.
Choose something you want to do but don't really care that much about. Be more ambitious than you think you could be, have a high bar on the outcome but purposefully choose to completely ignore the actual code. Make it write tests, dashboards, monitoring - whatever you need other than code to feel confident.
Now is the cheapest time it will ever be to experiment.
(You're welcome to make your substantive points thoughtfully, of course.)
Response: Try experimenting in a low stakes environment
From Google: "Exposure therapy is a psychological treatment designed to help people confront their fears, break cycles of avoidance, and reduce anxiety."
How does "low-stakes environment" fix that?
It's those who don't do it who'd need a talk..!
And I say this with the bitterness of someone having lost a big part of what made me enjoy this line of work.
Do you think it's better to experiment with high stakes projects? Do you think it'll be easier to avoid defaulting to line by line auditing of a project that is high stakes?
They've built an entire strategy around moving slow and doing it better
Code quality matters to AI just as much as it did to people
Temporal coupling: if you architect your logic in such a way that it assumes x always happens before y, and there's nothing absolutely enforcing that anywhere, and you then write new code without taking it into account.. now you have a bug
If you write code in such a way that it depends on a certain data shape (maybe in a string, which static typing won't catch) and you change it... now you have a bug
If you do ALL of these things all over the place.. now you have a fucking shitshow and might need a rewrite of your app. Lets hope Opus 7 can do it.
There are 1000000 things that can bite you that aren't "ooh that code is so pretty!"
The worst part about this is that if you aren't watching it, the AI will "solve" this with a thousand assert_3_plus_5_equals_8() test cases that just further clutter the workspace and the context window. Churn baby churn.
Even in relatively simple front-end code the AI messes this up all the time
If you let it go buck wild on a massive change, there are those times where it sort of paints itself into a corner but rewrites are cheap (for now), so it's just not that big of a deal, imho
Depends what you mean I guess. They're useful, but they're not accurate at it nor autonomous. In my experience Opus 5.5 can throw me 10 ideas after the fact, of which only 1-2 of them end up being good. Saves time but yeah
AI tends to add new wheels for a new feature in a large codebase, because finding, understanding, and generalizing the existing code to integrate a new feature will likely break existing features, cost more effort (and token) to implement and test. Human reviewers also tend to like these code since they are clean, easy to review, and get the job done, especially when they are not familiar with existing codebase.
The issue is that wheels like these accumulate in the codebase, and they inevitably entangle. When an upstream specification changes, when a new feature that works with many existing features are introduced, all these wheels need to be revisited, and in many cases they are not because the new test cases do not cover all the paths. The code base got larger and more complex, and contains more bugs than the unit tests can cover, especially when unit tests are often "bent" to fit the wrong behavior unnoticed.
Do what you want, write apps with AI and don’t review code, it doesn’t impact me. The claim that no one cares about good code is demonstrably false, and that’s all I’m objecting to.
so long as his software isn't used for anything that can affect you
I do, my coworkers do, my friends do, etc etc.
Similarly, once you pull in terrible AI slop code, the project becomes unmaintainable by humans and you need to maintain it with AI.
This is, of course, tangential to your point.
And if people are not looking it, those people don't care about their customers.
honestly i'm thinking it might be easier (and cheaper) to launch our own agent from GH webhooks and trigger `/code-review`..
If an engineer paid $150,000 a year reviews some code manually for about 30 minutes, the cost of their time is about $36.
How is this a proper product? Every time you open the desktop app the buttons may be on a completely random place
Worse, everything after the /grill-me is injected a second time into the prompt as a <command-args> so your prompt is literally added to the context TWICE. (first time in the original user prompt.)
I swear Anthropic is intentionally bloating your context so you burn tokens on useless bullshit like this.
good model
Of course, just like "line of code" is a worthless metric, "change merge per day" is;
for i in {1..10000000}
do
echo "I am working !" >> README.md
git add README.md
git commit -s -m "feat: working with AI"
done
Have you seen the benefit of the churn of 1000 PRs a day?
I don't want to use a product that shifts under me as I use it several times a day. I want a product that actually moves slowly and works very reliably.
As an end user: why would I care about how many hundreds of releases a month are being pushed if maybe 1% of them have features I care about?
Speed is not the product.
Honestly not sure that there are that many effective changes a day given how little complexity there is to their public facing product.
A random twitter said that, 6 month ago, it was 100% !
DECLINE ! SELL THE STOCK OPTIONS ! (wait, there are none, confused!)
Anyway, flagging as misinformation.