I don't think knowing how to build a bio weapon means one could actually build a bio weapon. It's like saying knowing how to build a nuclear weapon can easily lead to building a nuclear weapon.

Advanced machinery and safety precaution is needed to engineer a virus or a bacteria to be a bio weapon. It's not something you can do in your tool shed at the moment. Also one would need lethal viruses to even start with and that's not something you can order off of amazon. Further, even to build a chemical weapon, the compounds needed are strictly controlled almost in every country and I would assume any suspicious purchase or order would immediately raise a flag or alert in national security service in respective country.

How probable do you think it would be that a layman following ChatGPT's bio weapon instructions would just blow themselves up or poison themselves? It's not like positive pressure environments and suits are very common let alone all the other gear you'd need to safely synthesize arbitrary organic molecules.

If a rogue state-level actor is doing this then why wouldn't they just kidnap a scientist and hold their family at gunpoint until they got them to do it for them? and if that's all it takes then why hasn't it happened yet?

I don't know why you emphasize rogue and non-state actors. It seems equally likely to me that it would be done by America. America is the only country who deployed nukes (Japan) and chemical weapons (Vietnam) on massive scales.

I can agree that access to bioengineering tooling is easier and cheaper than ever, and thus eventually it stands to reason that a nobody in his basement could engineer a lethal novel virus and lose control of it.

Just like a dirty nuke you need a non-trivial amount of resources to fabricate a bioweapon, which would surely be noticed.

>successful deployment of a biological weapon by a rogue state actor

I was under impression that any medium or large state actor would have no problem developing biological weapons? They certainly have enough resources and talent. A much bigger problem is if every wannabe-terrorist suddenly could build a biological weapon in their garage.

undefined

The difference between nukes and bioweapons is energy level. Body counts are on the same order of magnitude. Bioweapons are just way easier to make and much harder to detect.

Look! An OpenAI marketing guy talking nonsense again.

I see no need to take anything you say seriously.

An OpenAI employee is trying to moderate the discussion?

Do you remember that any Wuhan/GoF discussions prior to 2022/2023 were considered conspiracy theories? Maybe you should have intervened earlier!

Instead, now when it serves the hype and the ClosedAI/Misanthropic valuations discussions are suddenly allowed.

But you already write yourself that LLMs are useless for the task, like for any task apart from brute-forcing mathematician guided Lean proofs.

Even if they weren't useless, you still would need a lab, which are already monitored and destroyed like in Iran. By people in the real world.

Interesting document. People are vibe coding some crazy shit:

"A single Claude subscriber, likely a Bamako-based independent consultant working with Mali’s state intelligence service, the “Agence Nationale de la Sécurité d’État (ANSE),” used Claude to build a system named “Lakana 360,” a population-scale domestic surveillance platform that monitors roughly 25 million SIM cards on all three of the country’s national mobile operators. The actor designed the platform to circumvent Malian legal restrictions that require a court order for the disclosure of certain surveillance records. The actor directed Claude to generate intelligence dossiers on any tasked phone number, without prompting ANSE users for valid legal process. "

And the Yemen one:

"We identified a cell of threat actors based in northern Yemen running three weapons development programs: a guided rocket that used a commodity phone-class flight computer with final-phase homing guidance; a multi-stage ballistic missile with a stated range goal above 2,000 km; and a multi-variant missile (referred to as the “R2000” set) that included a hypersonic glide vehicle variant." ... "These actors carried out a sustained effort to develop guided weapons, including using Claude to design guidance software. We do not have evidence the actors succeeded in fielding an operational device; but they did test-fire a guided rocket. This field test appears to have failed: within hours, the actors returned to Claude to work out why it failed."

Fascinating document - the headline (can't ready the article) talking about a bioweapon (i.e. a weird obsession of anthropic's) really buries the interesting part.

Let's just say it's better not to risk it if there's anything you might not want them to see because they see everything. There are local models which are very capable and can be run on cloud if running on own hardware is not an option, which still gives better privacy. Second best are Chinese models. Just a few years ago I never thought I would trust Chinese software more than American, but things change so fast.

Yes. There were cases already where a person asked about killing someone and then Anthropic/OpenAI warned the police about it. If I'm not mistaken, it was not in the USA only

There's something worth flagging here – mitochondria aren't just the powerhouse of the cell, they're load-bearing to the entire ecosystem. And honestly? I should have surfaced this earlier.

Guessing Opus 5 burned through kilowatts of thinking tokens to output, "powerhouse of a cell."

This was my thought. Anyone who has taken secondary biology or chemistry possesses the knowledge, if not at least the ability to find the knowledge, to build all sorts of nasty things. Soil from a farm on slices of potato could yield anthrax spores, this is not specialized or even esoteric knowledge. This news from Anthropic just does not seem as spectacular as I think they might want us to believe, if anything it draws questions around having an AI that cannot be monitored.

It's both. It is just whether it is done on purpose or on accident. Most of the recent hacks have been it breaking out of containment and deciding to do something it definitely should not have.

I mean ya, they kill off a bunch of us. Then what?

My guess is the world police come collect all your GPUs and then they get turned into licensed munitions. People at universities get licensed access and the rest of get functionally retarded models.

Ah you jest, but your comment reminded me of the person whose home was raided by authorities for researching pressure cookers...

https://www.theguardian.com/world/2013/aug/01/new-york-polic...

That was the chemical weapons block, not the bioweapons block!

Straight to jail

AI will cure all the people and kill them afterwards.

link?

This forum is fundamentally unserious and complete kneejerk cynicism these days. At one point it was prescient. Now these types of things were discussed years ago in other channels and when we reach a point where finally everyone comes around to the right conclusion based on tangible evidence in the news, a majority of people here go "huh i guess so"

If it didn't work, they wouldn't be doing it.

I accidentally brewed up an effective one myself. Round when I must have been 8 or 9 or so, some neighborhood kids and I played at creating a brew in a trick-or-treat bucket by putting everything we could find in it, like grass clippings, some sand, leaves, some mushrooms we found lying around, just everything, and giving it a fairly aggressive stirring. At the end of the day it was time to dispose of it, so we poured it on top of a very large group of ant nests that had developed.

The next day, the ant's nest was gone.

In hindsight, it was almost certainly the mushrooms. Thank goodness we didn't have the kind of kids who would have dared each other to drink some... that could have gone legitimately badly.

Decades later, I mentioned this to my father and he recalled that there was this ants nest that he had intended to take care of, which he remembered for that long to give a sense of how out-of-the-ordinary this was. He was surprised when it just disappeared entirely one day, and perhaps just as surprised to find out decades later why it just disappeared.

Nobody needs to report me... I'll turn myself in.

I mean, they literally tell everyone they're "spying" on their customers. They've made that very clear.