"On the other hand, a missile with a man or two within, controlling flight by graphitics, would be lighter, more mobile, more intelligent. It would give us a lead that might well mean the margin of victory. Besides which, gentlemen, the exigencies of war compel us to remember one thing. A man is much more dispensable than a computer. Manned missiles could be launched in numbers and under circumstances that no good general would care to undertake as far as computer-directed missiles are concerned . . ."

Now we know they won’t even be able to read and write. But maybe Claude sonnet 12817 can maintain the operating system. It’ll be 17,000 dependencies in and mostly written in JavaScript, but it’ll work.

Which doesn't seem like it. The ideal is to keep removing couplings. But in practice, I find that focusing on the statefulness, the distribution-specificness, these "accidentals", is how you actually get leverage.

The obvious social consequence is now the tendency towards centralization, tyranny, "you people can't think for yourselves, not because you have some kind of 'inherent intelligence defect', but because the world is so complicated and interconnected that only coherent systems thinking (conveniently administered from one central point) is what can save you". In the Enlightenment world, the ideal was that every man could in principle, think for themselves.

A good ending might be that everybody benefits each other by distributing intelligence, but in practice centralization always seems to win.

I dare companies to gamble on running their businesses on absolute skeleton crews long term

Most humans aren't useless by any measure of most or usefulness. Our culture is our tools, dying due to a lack of antibiotics doesn't mean you didn't have anything to contribute, it means you didn't have access to the tool you needed. This is the problem we need to guard against.

No one is going to lose the ability to reason! This is accepting the most extreme version of the marketing presented by frontier labs, don't do it. People may lose fluency or familiarity or access to training on specific skills. We should put some care into making sure that can't be a catastrophe, for sure.

The titular Pump Six breaks down and basically there's nobody left in the world who knows how to even reason about the steps needed to fix it. There's just some log dump and nobody understands what it means. Humans are no longer capable of complex reasoning, they just live in the moment and hump.

PRs shouldn't really be reaching 8k diffs no matter who authors them.

(but yes if AI is going to propagate, they need to fix their uptimes)

Everyone must code alone

If you listen closely, you may hear them squawking in annoyance after each announcement.

The Samsara of AI subscriptions.

Would be interesting to see how OpenAI and Anthropic would respond to being "accidentally" hacked by their respective Frankenstein monsters.