> The online retail giant said there had been a “trend of incidents” in recent months, characterised by a “high blast radius” and “Gen-AI assisted changes” among other factors, according to a briefing note for the meeting seen by the FT.
> Under “contributing factors” the note included “novel GenAI usage for which best practices and safeguards are not yet fully established”.
> “Folks, as you likely know, the availability of the site and related infrastructure has not been good recently,” Dave Treadwell, a senior vice-president at the group, told employees in an email, also seen by the FT.
Also some SVP over there: '"folks", we'll measure your performance and bonus based on how much you use Gen AI:)'
GenAI is literally the direct reasoning they used for laying off 30k people.
> “As we roll out more Generative AI and agents, it should change the way our work is done. We will need fewer people doing some of the jobs that are being done today, and more people doing other types of jobs,” [Amazon CEO Andy Jassy] bluntly admitted.
But what is reported? Management firing people? ML. Engineering screwing up the uptime? ML. Someone screws up their job? ML.
Don't you know? ML is killing people in Iran today. Not mullahs. Not the military. ML. Obviously that's where the responsibility lies ...
Usually blaming ML is like suddenly coming up with conspiracy theories like here, or impossible suddenly added requirements, and usually utterly ridiculous ones (like criticizing Deep Blue for not being able to play poker, yes I realize I'm old, but it's a bit like criticizing the very best competition canoe on the planet for it's disappointing spaceflight capabilities)
Like here: large blast radius AI-assisted outages ... we've all written software, and we all know the problem here: THEIR TESTS SUCK. Probably because they fired all the good SREs for insisting software teams spend time on tests, or demanding integration test failures are fixed before shipping the software.
By the way: I'd like to point out that in most/all industries where jobs are lost on a large scale the situation is like the Amazon situation: ML is not even remotely involved. So while I get the criticism, it doesn't work like that. The Auto industry first got blasted with very traditional engineering, which worked and depended on very old style mathematics. What's happening in factory automation is 99.9% 3d geometry (to the point that ML, is actually a simplification of the problem). Then the auto industry got blasted with what every industry got blasted with: stuck in demand-limited markets. Every car company can easily build 10x more cars next year, but there's no point: nobody will buy them. So the only thing worth doing for these companies is to produce cheaper ... and that means getting rid of people (when end-to-end taxes on income in Europe are 60-85% and actually rising). With only a few exceptions, these companies find ML too expensive for projects.
So while I understand "we're defending our jobs", it's misguided ... the big job losses in the west have nothing to do with ML. MAYBE those are coming, but large job losses have been predicted in the last 50 AI "revolutions". 49 times that was wrong. And the actual problem is really a return to 99.9% of history: when it comes to doing what is needed to keep society going 10%, maybe even 1% of people can do it. That means you need something for the other 90% or 99% to do.
The solution is the only thing that has helped in the past: having the government put on huge public works. From building the pyramids to the Sagrada Familia (and yes, wars. But let's please not do that), or ridiculous engineering projects like Europe and America's rail networks. There's a stable in the Italian alps that has a private rail connection. So fix the problem. I don't know: build a large cathedral in Washington or something. Hell, hire people to make sure it has a depiction of the last supper where every square micrometer of the painting was designed by an AI with 1000-member engineering team, so people can spend their entire life looking at the painting with a microscope and find new details every day. Let's do something "great", in the sense of an enormous effort. Fly 100 missions to Alpha Centauri. Fix the demand-limited issue the economy has. "Do more with more". And stop blaming ML. Hell, I'm currently in an old European city filled with 200-year old buildings. Quaint. Cool. Except ... not really. 90% of these buildings suck. Can we just rebuild 95% of ... all European capitals? Every building that is way too old and has no reason whatsoever to be preserved other than it's currently slightly cheaper ... can we please just rebuild them better? Do stuff like that.
If it works for models, why not humans? /s
But don't tell anyone --- and if you do, don't blame AI because it's all the humans fault for not shaping their questions in the "right way".
0: I think this is the eras cowboys win so they're (unsurprisingly) smart about doing this
I remember having to do such a code review before an AI in a highly complex component, and it would take a full day of work to do it. In this day and age, most of the people i know take like half an hour and are mostly scanning for obvious mistakes, where the bigger problem are those sneaky non obvious ones.
What could work - llm creating a very good test suite, for their own code changes and overall app (as much as feasible), and those tests need a hardcore review. Then actual code review doesn't have to be that deep. But if everybody is shipping like there is no tomorrow, edge cases will start biting hard and often.
Beatings will continue until senior engineers leave?
Another case of AI as a scapegoat.
But what they’re missing is all code quality is going to tank, and we are just going to accept that. Just as artisanal goods were replaced in the Industrial Revolution with mass produced inferior ones.
People will accept bad code if it is cheap enough.
We’ve gotten used to aiming for great, even if we often only hit functional. The new bar is going to be so much lower. Welcome to the era of cheap bad code. Lots more software, lots more value overall, but much worse reliability. Every day the apps I use get buggier.
Machines that make e.g. paper are great. They are immensely more efficient, but extremely consistent and superhuman (try making that perfectly smooth letter paper by hand).
Human written software is the same. Where you had N people copying data from spreadsheets for M suppliers into an internal database or whatever, you now have one program doing it. It can be scaled infinitely for a fraction of the cost. It _never_ messes up. The cost of the software developer is trivial in comparison. Software was a space where the marginal cost for quality was extremely cheap.
I don't get how AI fits in here. Software already had massive scale. You aren't replacing a massive data entry team with AI, you're replacing a reliable piece of software written by a human with a reliable (?) piece of software written by AI controlled by a human. There's no increase in scale. Until the reliability issues are fixed a very noticeable decrease in reliability (sure, some software was bad already, but now the good developers are also writing bad code).
This doesn't seem like a natural step to me at all. The best explanation I can come up with is AI is just being used as an excuse for destructive penny pinching.
This is the core insight for most businesses.
When evaluating the impact of AI on velocity, the first thing to consider is how long it takes for a one-line code change to get into production, including initial analysis and specs.
You can't get faster than this.
Being able to easy create apps means huge supply, which means commodification of software just like the commodification of physical goods. Mass supply means low prices. It won’t be economic to have artisan coders any more than to have artisan goods makers.
Safety critical engineering and infrastructure layers will (eventually again) be rigorous. Everything else is headed to slop.
My craft died. I’m sad. Time to move on.
Also, mcd ain't at the end much cheaper, just marginally, the choice of drinks is pathetic, usually no beer. The main reason folks go there because its easier/faster than getting table in real restaurant. But also the environment in mcd is absolute soulless cheap fugly shit. (there are kids corners to be fair, but they are often disgustingly dirty).
Its a very good analogy at the end IMHO, maybe just not tilting the way you intended, at least not here.
That is, ah, very much not the case for AI slop.
Quit when the work is doneFor instance, I want to engineer [more]. Closer to management or sales due to scope creep. At this rate, by career-end, I'll be operating a small country by myself.
It's a bit hard to believe this.
Lol. Lmao. You have got to be joking. Seniors leaving in droves is how that plays out.
I know there are some companies that never did code review, but this is Amazon. They should know better.
Amazon’s ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a “deep dive” into a spate of outages, including incidents tied to the use of AI coding tools.
The online retail giant said there had been a “trend of incidents” in recent months, characterised by a “high blast radius” and “Gen-AI assisted changes” among other factors, according to a briefing note for the meeting seen by the FT.
Under “contributing factors” the note included “novel GenAI usage for which best practices and safeguards are not yet fully established”.
“Folks, as you likely know, the availability of the site and related infrastructure has not been good recently,” Dave Treadwell, a senior vice-president at the group, told employees in an email, also seen by the FT.
The note ahead of Tuesday’s meeting did not specify which particular incidents the group planned to discuss.
Amazon’s website and shopping app went down for nearly six hours this month in an incident the company said involved an erroneous “software code deployment”. The outage left customers unable to complete transactions or access functions such as checking account details and product prices.
Treadwell, a former Microsoft engineering executive, told employees that Amazon would focus its weekly “This Week in Stores Tech” (TWiST) meeting on a “deep dive into some of the issues that got us here as well as some short immediate term initiatives” the group hopes will limit future outages.
He asked staff to attend the meeting, which is normally optional.
Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added.
Amazon said the review of website availability was “part of normal business” and it aims for continual improvement.
“TWiST is our regular weekly operations meeting with a specific group of retail technology leaders and teams where we review operational performance across our store,” the company said.
Separately, the company’s cloud computing arm — Amazon Web Services — has suffered at least two incidents linked to the use of AI coding assistants, which the company has been actively rolling out to its staff.
AWS suffered a 13-hour interruption to a cost calculator used by customers in mid-December after engineers allowed the group’s Kiro AI coding tool to make certain changes, and the AI tool opted to “delete and recreate the environment”, the FT previously reported.
Amazon previously said the incident in December was an “extremely limited event” affecting only a single service in parts of mainland China. Amazon added that the second incident did not have an impact on a “customer facing AWS service”.
The FT previously reported multiple Amazon engineers said their business units had to deal with a higher number of “Sev2s” — incidents requiring a rapid response to avoid product outages — each day as a result of job cuts.
Amazon has undertaken multiple rounds of lay-offs in recent years, most recently eliminating 16,000 corporate roles in January. The group has disputed the claim that headcount cuts were responsible for an increase in recent outages.
Seriously, who even cares? It’s probably going to be “guys be careful but also continue to push slop kthx”.