This has already been discussed heavily in this thread:

https://news.ycombinator.com/item?id=44522772

Link to the full paper: https://metr.org/Early_2025_AI_Experienced_OS_Devs_Study.pdf

Overall the study is a very small sample size (16), with mixed AI tooling and mixed AI experience. It's an interesting data point, but honestly not an extensive enough study to make any causal determination. It's certainly plagued by much of the discourse around AI being highly polarized, as well as AI being such a broad category as to have little meaning overall.

Quoting from the above thread:

> My intuition here is that this study mainly demonstrated that the learning curve on AI-assisted development is high enough that asking developers to bake it into their existing workflows reduces their performance while they climb that learning curve.

The above quote, very much matches my personal experience. The first month or two was very hit or miss, and plagued with frustration. As I got better using the tools, and figured out new workflows, and settled on better tools, it's become a much better experience for me. Specifically, asking ChatGPT or Claude to generate a function for me sucked, editor tab completion with a good model was better, but still occasionally frustrating, chat in cursor was better than that, and claude code as an agent has been fantastic. But the journey was long and required a lot of reading, video watching, and listening to podcasts about how people who are successfully using AI coding tools work.

Currently I feel like I'm about 2x as productive (note: I'm not a particularly quick developer, so YMMV).

This is exactly my experience as I started heavily using LLMs for coding. It can feel like a trap,Im sifting through all the generated code instead of reading the docs and finding the correct way to do things, because I expect the machine to output the answer, I spend a lot of time prompting.

When it works on the first prompt its magic. I especially like to generate UI components, but for more complex things its a major time waster. Often complex functions just dont work and debugging is slower than rewriting it from scratch.

Ironically, AI tools can make you slower if you rely on them for complex tasks without understanding the internals. You end up spending more time prompting, debugging, or reverse-engineering the generated code than if you just wrote it yourself. This tradeoff is especially noticeable in open source work, where maintainability and correctness trump speed. AI tooling still requires substantial human judgment.

I believe this but there is another side of it where it doesn’t feel as tiring. I have more energy left after a longer AI session than a shorter traditional one. That’s worth a lot.