"One occasionally reads newspaper accounts of how two programmers in a remodeled garage have built an important program that surpasses the best efforts of large teams. And every programmer is prepared to believe such tales, for he knows that he could build any program much faster than the 1000 statements/year reported for industrial teams. Why then have not all industrial programming teams been replaced by dedicated garage duos? One must look at what is being produced."
Brooks argues that the coding that happened in the garage is only a fraction of what you need for the final product:
"It costs nine times as much. But it is the truly useful object, the intended product of most system programming efforts."
What we're seeing right now does not feel much different. You can greatly accelerate a first draft, a greenfield project, an experiment, or at least the initial iteration of one. Changing a live production system or a widely used library is an entirely different proposition.
Low risk work is tremendously accelerated. Just look at the memes that people re building. One of my favorite right now are derivations of this thing: https://x.com/jamesonhodge/status/2118733265372860826
People are running away with it in hilarious ways using all sorts of storyboarding, 3d environments, gameplay, etc etc.
I work in a highly regulated environment. My AI spend is ~$300/week and it has been tremendously helpful in troubleshooting incidents(!), increasing test coverage, gaining understanding of legacy/inherited code, applying fixes, etc. But shipping new features is something that impacts us in many many many ways and the uptake there has been quite slow.
As we’ve seen happening in the open source projects with the piles of bugs being reported to them so much of what’s going on doesn’t seem to be new features, but just longstanding technical debt coming to the fore.
Maybe AI can’t write that new feature for us really fast. Maybe we asked. Maybe the code was complete garbage and it made it clear we really don’t want to keep piling on top of it and need to fix the issues we’ve known about for a long time. Adding new features at the old pace was a problem. We can’t handle the new.
Between that and the fact that some kinds of tech debt are much easier to pay off now that’s where a lot of our “new capacity” is going. Shoring things up.
I am doing this now with a massive internal app. Our internal customers see this as a drop-in performance because one of the senior team members, me, is not adding new features. But we are addressing a massive technical debt in about one-fifth the time it would've taken without the use of LLMs.
However, for those that are actually curious, the submission is not itself a cynical or dismissive take. The question in the title is interrogative and contextual, not rhetorical and universal, and the presentation tries to break down some of the bottlenecks and challenges that are leading to AI integration having a different impact on software engineering output than some people might expect.
Rather than suggesting those bottlenecks/challenges are universal or intractable, it just enumerates some of them and presents some survey analyses on how widely encountered they are.
It's actually pretty interesting and the kind of thing that engineers and those who work with them want to be reflecting on, so that they can better engineer their usage and processes around AI itself.
edit: if you're seeing a slow down in PRs, it's more likely that 10 of your 10x'ers are slinging code in 3-4 plus accounts in codex on the side during their shift on 8 startups.
I would want to see more cloud providers embracing fully-autonomous deployments for new users, I created this benchmark to track progress here: https://github.com/cloudishai/ai-deploy-bench
Either watch the presentation if you want to spare the time (a big ask even at 2x speed) or move on.
The presentation is about the results from an actual investigation on why a particular group only saw modest improvements in shipping. It will probably be more relevant to groups more like the group studied (a large dev org) than others.
The general idea is that there is a process pipeline between code change and shipping, and LLMs don't necessarily speed up all the steps at the same rate, especially in large, mature products.
I think people already have forgotten how long and arduous software development was just a few years ago.
The biggest bottleneck now is CI/CD/Experimentation with respect to engineering, and product alignment with respect to the business.
I have seen it among “the general public“. I’ve seen normal people complain about things in open-source projects or other contexts where to whatever degree they accepted programming took time before now they think it’s all just a magic wand with Claude, no matter the level of complexity.
I haven’t heard a single person at my company in management at any layer complain that we’re not working fast enough now that we have AI. One of my friends has 100% told me their management is pulling that. The management over there is on the everyone should do four times as many tickets, and use at least X amount of tokens per day. otherwise you’re slacking off train.
It’s all in who you talk to.
Engineers that have only started full time work post 2023 haven't experienced the counterfactual of not having AI.
They see all the mundane business/eng bottlenecks that are pervasive now and don't truly know how much better it is now than before
Was it? Behind any arduous development process, I'm ready to bet there's a very inefficient communication channel that has nothing to do with code or any technical matters, but still impact productivity. Think how hard it is to get a Definition of Done alongside the Change Request even in the business jargon.
> The biggest bottleneck now is CI/CD/Experimentation with respect to engineering, and product alignment with respect to the business.
It's like building crooked houses, then saying that the biggest bottleneck is inspection. Do a 5 why[0] around the "bottleneck" and the answer is likely not there at all
That is what I'm saying though. Shipping to production is mostly bottlenecked by product/business leaders who haven't described a clear vision/communication of what is wanted.
I've been a SWE for far longer pre coding-agents than post. The actual development of software was arduous. So much of tech was/is a long list of heavily scoped down tasks that eng takes an order of magnitude longer than today.
A quirk of coding agents is how they assess timelines. Ask it to scope a project you want done and it will answer in engineer days from pre-ai days. In actuality it can be done with a prompt and a few hours in claude
I disagree. Based on my personal experience, a lot of time is spent on fine tuning code and double checking the docs. The first pass can be very quick especially if you already have sample code (from the docs or other parts of the project). But when it came to fine tune, that's where the other 90% of the time is going.
A lot of daily activities of engineering is massaging code, either from bug reports or requests after someone tested the features. And a lot of time is spent gathering details, comparing it with what already built, and then apply correction.
From what I've seen with people doing agentic coding, that feedback loop in the development phase is very superficial. Then the most important feedback loop (the one with the user) is distorted as the software is constantly changing.
Personally, we’re just beginning to ship faster because we’re a big ship and turn slowly. We’re still getting up to speed but we’re also bogged down with bug fixes, security fixes, and the review bottleneck. Reviews are especially tough now that every commit is much larger because it is now much more correct than the crap we were submitting before AI.
Also, many people are simply lying and hate AI. There are people I’ve encountered who look for any evidence to justify not embracing it. It’s frankly tiring but I understand some people cannot handle change and are frankly terrified of what is happening to their careers and their concept of self-worth.
If you want to drive at high speed, you need to build a highway first.
At least at Block our velocity have gone way up. One big unlock was designers building their own features not making it a developers problem.
Our biggest bottleneck is PR reviews but we are also gradually solving that.
That may or may not include shipping without reviewing code. Something I think is a real option to consider as LLMs and agents become more capable.
The reality is, any point where a human is involved is now a bottleneck. Even when that human is using an agent to write code. This will apply to the review process as well.
It feels this way, doesn't it? At least until we focus on something besides pure efficiency.
We are. ~300% as of late-summer when I calculated. Models have gotten markedly better since then.
At least in entertainment I only see cancelled products rapidly increasing.
Specifically, I was able to consolidate some CSS because I could ask Claude to calculate the cascade before and after and verify that nothing had changed about rendering.
I’m sure there was a way to do that before by scripting a headless browser, but I didn’t have to learn that tool to unlock this capability.
> Customer email, 9:42 am: "I have a partner asking for our SSH2 public key. I tried using Tediware help but eventually it said to contact support. Are you able to assist?"
> Response, 10:47 am: "You need to generate a key pair yourself and keep the private key. Tediware does not issue SSH keys. Here's what you do. [instructions follow]"
> Follow-up response, 12:20 pm: "Generating your own key pair is now part of the app."
> Time from support request to new feature designed, coded, tested, deployed and announced: 2 hours, 38 minutes.
Side note, this feature has been solid in prod ever since (that story is a couple of months old).
Obviously, some things take weeks still, but they're hard. I just finished writing an X12 translator/parser, from scratch, in Rust. The old translator/parser was written in Ruby and ran documents at 175 kb/s. The new one handles 1 MB/s. It took me seven weeks of on-again/off-again dev work, fuzzing, testing, etc. It went live in prod yesterday.
What about risk assessment? Are the customers guinea pigs?
I think it would be more widespread if there wasn't as much pushback against AI. Big Walk disclosed they used ai to fix a bug and it immediately got review bombed. Kids on Roblox don't care as much, so you can see just how transformative it has been there.
Personally I think Steam should either get rid of the label or expand it to any usage because the carve-outs are arbitrary and pervasively widespread in use - I don't find it believable House House's developers are still almost entirely coding by hand and only leaning on AI for the hardest problems lmao. The stigma isn't sustainable once full disclosure is normal, and then games can be judged simply on their merit either they're good or bad.
Failing that it's hard to imagine what IGF looks like next year because everything currently in development will be using AI in some fashion, either they're going to have no candidates or no honest ones.
> “Programming is a creative discipline whose labour deserves the same respect we extend to any part of making a game. This extends to the past work used to train these models without the creators’ consent.”
https://www.videogameschronicle.com/news/big-walk-studio-say...
“Sure there’s a massive increase in the number of apps on the app stores, there’s new examples of decompiled and recompiled games popping up all the time not to mention all the vibe coded games, many companies are reporting they have new bespoke vibe coded tools to do some task they used to do manually, people are just cloning popular apps like photoshop with bespoke customisations to their needs..” etc.
I already know the tiresome response to any such example given by people of this mindset. “Yeah but those aren’t TRUE examples of users getting features faster!” Whatever.
Google Search: 1998
Gmail: 2004
Google Map: 2005
Google Docs: 2006
Chrome: 2008
vs
Claude 3 (+ tool calling, web search): 2024
Claude Code: 2025
Claude Design, Sheet, Data: 2026
"We" (whatever it means) are shipping faster now, no?
Pre coding : Requirements, grooming, solutioning (cross product specially), design, LLD
Post coding: Cross product testing, E2E integration testing, CRQ approvals, Deployment, pilot, roll out.
SWEs don't just do coding. AI has only solved that and testing to an extent.
I also see companies that don't know what they want to build, suffer from toxic culture etc -- they are not faster because their core problems never had much to do with software development.
Sure, there are plenty of developers clinging to the "old way" or using AI in ways that add complexity and slow things down but I don't see this as the dominant group.
Claude Code TUI, probably the poster child for AI-driven coding, itself is still going to sh*t.
But does it say what quantifiable evidence they have to show software teams are NOT shipping faster now?
More to the point, the price of RAM that has gone berserk. Easy solution, let's then make our OSes, browsers and almost all the relevant software that we use day to day an order a magnitude more nimble, for example if a Chrome instance now eats 1-2GB of memory then make it eat at most 100-200MBs of said memory while doing the exact same things (if not more). The much lauded AI revolution in code-writing hasn't created that, not sure if it will ever do, it will just add slop after slop after slop, in fact it wouldn't surprise me if the memory requirements for something like Chrome won't increase as a result of the recent vibe-coding mania.
But by this point the AI zealots are already running the asylum, so nothing to be done.
The sad thing is those using AI to code (and I certainly do) don't realize they're producing more but just more of the same turds they were already producing.
And in addition to that they're producing turds nobody gives a shit about (ah!).
Where, indeed, are the both less memory-hungry, less CPU-intensive and more secure rewrites of the pigs we work with on a daily basis?
Nowhere to be seen.
Because the AI zealots are way too busy adding crap features over crap features.