Productivity at tasks: yes — it built 2 full products from scratch, ran DB migrations, wrote SEO pages, set up analytics, deployed everything on its own.
Productivity at outcomes: no — still at $0 revenue after 18 runs. It kept optimizing things nobody was using.
The key insight: AI productivity is massive when the task is well-defined. But left to its own judgment, it tends to choose building over selling, and polishing over distribution. Same trap junior engineers fall into.
Full writeup: https://dev.to/wpmultitool/my-ai-agent-has-been-trying-to-ma...
For a more concrete example, I have an interface to the data that comes from every sensor on the oil processing facility. It has a built in "AI" (I try not to use that term!) but it has a feature where I ask how to process data in plain language and it'll give me the calculations, then it'll also provide a plain language summary of all the calculations I conducted. That saved me 10 hours of work.
I am a negative nancy on LLMs in general but I still passionately believe that they're a tool which every white collar employee will need to learn to use effectively.
I cringe when I hear engineers say "I didn't know the answer so I asked ChatGPT" but I also do worry that I could be significantly outperformed by another engineer with 10 years less experience in engineering and 1 year more experience in judicious use of LLMs.
I use it more and more as advanced API documentation and writing code snippet. It made me a little bolder, and reduced friction from coding. I will test Claude Code in the upcoming weeks.
I used to hand-write a lot of the scripts we used to automate processes (mostly Google Apps Scripts and Python). It was very helpful to us but slow going and sometimes frustrating.
In the past, I was limited by my programming knowledge and ability. Now I am limited by my knowledge of our business and ability to explain what I want to happen.
With LLMs, I can generally 1-shot most things I want to automate, including things that would've taken me days to figure out in the past. I generally just say that I'll spend 30 minutes on something and more often than not, it's either done or very close at the end of that time.
1) Debug mode from Cursor. It highlights all possible hypotheses based on the steps to reproduce and whatever you know from the code. It slaps down logs and tests all these hypotheses at once.
2) Log reading, though this is a variant of the above. At times we get massive logs, like 10k lines for a bug. We go through what may possibly cause the bug - what screens, threads, race conditions, improper handling of ANY_KEY or default or something. Then we ask Opus to compare the logs to see if it matches the story. I ain't manually checking Line 314, Line 500, Line 44 to see if it shows in logs, but AI is just great at piecing together this whodunit.
3) Finding documentation. So there's been a few bugs unsolved around how very specific Android devices handle wifi reconnection or bluetooth or Ethernet. Instead of making assumptions, I can dig into the AOSP source code directly and see the conditions that trigger these. AI will even point out the exact lines.
I can now do everything by myself on most projects. Up to ones that would have taken at least 2-3 other people before. Before I would have had to delegate it just because I couldn’t do it all by myself on time. I know how to develop (professionally did 30 years) and I know cloud (professionally for 8 years). I just didn’t have time.
On the other hand, I haven’t done web dev in a decade. But I can vibe code an internal website for operations and authentication via Amazon Cognito. It’s just a free feature that I give them even when it’s not in the contract
But let’s say I make more as a staff consultant considering my actual billed hours, benefits (401K, health insurance, paid time off etc) than I would make working independently and I always know how much I’m going to get paid.
But I will say in today’s cloud consulting environment, it is a race to the bottom unless you can lead projects and even then there are relatively few high paying (over $200K with benefits and 80% utilization not including PTO) outside of being a full time employee working in the consulting divisions of AWS, Google (they had an RTO mandate so I ignored recruiters) or maybe Oracle.
The only reason they can justify American rates for someone like me who knows cloud but who is mostly a developer is because I can lead projects and churn out code quickly - thanks to AI.
We have software written in what is essentially an obsolete platform (a RAD solution from the 1990s). We've been slowly hand rewriting it into React. A single developer converts one a week by hand, and the results are good (we still have over a hundred).
We've developed an internal tool to turn the software's definition files into a single XML, and then are feeding it through a multi-process Codex pipeline (multiple instruction files, that product intermediaries), which ultimately outputs a "90%" working React page.
Our developers then PR the output, fix/adjust/test, and release. We've gone from one per week for a single developer, to roughly three per week without a marked quality loss.
It is taking care of most of the repetitive parts of the work, and allowing our developers to spend more time in technology they're familiar with (React) instead of legacy stuff. Is it perfect, no, but the cost/benefit is clear.
Doing X in Y days does not mean much, same thing happened with Ruby on Rails.
Even if the processes were identical, using an LLM arguably likely broke any privilege and confidentiality protections you might previously have enjoyed. See, e.g., United States v. Heppner (S.D.N.Y.) (recent case of first impression).
So there's that.
I was also going to comment separately that LLMs have been great to identify opposing counsel whom I suspected of not litigating in good faith / actually reading the cases and applying them. Now the confabulated cases are proof positive!
As long as it is not a privacy sensitive, high stakes or human factor related matter, I choose LLM any time of day.
All the bullshit lawyers made up to protect their own job I deeply can not care less about.