Re coding - the code that is written by a wide audience of unqualified individuals is not useful code. If you aren't an engineer and you prompt a model to build something, you can't attest to its reliability. (Moreover, anyone could have prompted for the same code.) Until it can be verified that it actually works, it's basically just slop. And such verification is meaningless without the person having the capacity to take responsibility. If you are not a civil engineer, it doesn't matter how much you prompt AI to check your designs; you can't reasonably sign off on them. In practice, most people prompting AI are taking zero responsibility. OpenAI and Anthropic's hacking scandals are a really wonderful example of this.

Moreover, in practice, most of the software being built is probably not useful. There's a nontrivial chance it does something bad, so you can't use it without sandboxing it. It doesn't mean no useful software will be built; I'm saying that much of the software being built is unreliable. In fact, AI almost certainly decreases the number of people who are even qualified to judge software quality because everyone is being deskilled since they are allocating their thinking to the model (or in the case of current middle/high school and college students, never learning to think in the first place).

The dumping of slop into ecosystems that previously had institutions of trust in place is a HUGE harm. For example, these folks are overwhelming academic publication systems and the internet broadly with their slop. Cyberterrorists will do the same.

Even if tomorrow AI satisfied the absurd criterion of being able to only produce reliable things (if you prompted it accordingly: "Be reliable!"), you'd still have the problem that a cyberterrorist could come along and say "reliably hack X". The space of bad acts is far bigger and more accessible than good ones. This is what I mean by exploiting / harming the commons.