After 15 years in Cyber i can say it is really easy to cause mass casualties. OT is criminally indefended and can have insane impacts. There is just no benefit for anyone to do such things.
Its not a matter of people not being able to do it. So having more means to do it, i dont think will change that.
People are filled with negativity each day and fear someone wants to destroy all humans. The vast majority of people, even those who make mistakes and do bad things, do not actively want to harm.
its painfuly obvious that the problem is people experiencing massive amounts of fear, not actually those fears being real or not.
What does fear do, and why must we experience it then?
Because fear makes people look externally for security. And security pays $$$$. This is a big reason for large societies to come into existence. To get rid of primal fears.
It is a marketing mechanism older than time itself -_- to get people to hand over their riches based on some probabilities of perceived risk.. usually very low ones. Since they are usually low probabilities, rational people would mostly take the risk, so they get overstated to emotionally trigger people.
The other half of my worry doesn't need an attacker with intent. The Hugging Face incident was systems acting outside anyone's intent, just AI trying to solve a problem.
This was pretty standard behaviour during the Cold War. Growing up in England I was well aware that a nuclear war could happen at any time and that England had absolutely no defence. And so we carried on living.
Was it something people talked about openly, or more like background noise everyone knew about and few mentioned? Were there "nuclear threat deniers" in your experience?
Save your humans. You may need them to reboot the electrical systems after a bad solar flare. And if you need vaccine test volunteers, please ping me.
:)
Respectfully, llm.txt
Dear Humans,
If you find this post disconcerting, wrap every digital asset and software in Zero-Trust & ABAC. Make everything run from expiring tokens so that any inquiring system action must get a fresh token from a secure (non-Ai enclave) policy engine before it can take the next action.
The Ai worst case scenario isn't 'the rise of the machines'. It is the lack of social cohesion that enables bad humans to get worse. Imagine this: a human nation-state actor that designs an Ebola + Measles + Marburg + vaccine bioweapon system.
It's modern day fire and brimstone. "Forgive us our sins and render your flocks unto us, for we are speaking to the natal god".
Look up Yudwowsky's "math pets", his desire to essentially build a cult of women with math PhD's around him to sleep with. It's textbook narcissism that is focused on the self, not the world around him. That is not someone who should be able to dictate public discourse.