Also, while I don't believe in the doomsday narrative.. hey maybe don't do things that are functionally useless but seems designed to create this doomsday in the first place? Like you can't say "oh we're so scared" and do stuff like this without completely losing your credibility.
Step 1: Spread nonsense FUD about open-weight models.
Step 2: All of the clueless mainstream media sites repost it (e.g. CNBC, Reuters, etc.), regurgitating Anthropic's talking points.
Step 3: Equally clueless politicians read those mainstream sites and get scared.
Step 4: They contact Dario to ask about it.
Step 5: Dario tells them, "well, you know... this is only a problem with open weight models; if there are no open weights to replicate then this cannot happen".
Step 6: ??? (I'm sure you can imagine)
Step 7: Profit.
8B models aren't very good, but there are some parallelizable workloads which would benefit from a surface area as large as here. I would think database search is parallelizable in this way.
Your fridge is searching for needles in haystacks.
In particular, I've seen a lot of people say that our response to things like the Hugging Face swarm should be to prosecute whoever chose to run the swarm. If this kind of replication becomes possible, that won't work, because we will have to deal with swarms that nobody chose to run.
That rule has served me well enough!
If you won’t take that bet, it’s irresponsible to ignore the possibility.
We are truly in the brute force phase of AI. Like a database without indexes, we are doing full table scans to pick 2 rows.
On the other hand, a self-replicating and adapting AI worm does sound legitimately pernicious. Not world ending, but possibly an end to the online world.
They talk about distributing layers across devices, but then you'd have communication on the network and hours of driving the devices full tilt to get through even a single turn.
There's so much that can be monitored to catch these, even if we assume that the worm could hack the OS's usage monitors, it can't hide the heat and power draw.
Datacenters and PCs would adapt, requiring better monitoring than the AI companies' method of letting things run blindly for days/weeks.
IoT would be tougher, but it is also the weakest hardware, sometimes battery powered, thermally constrained, lacking in memory for processing even one layer or lacking the necessary connectivity.
In the bizzare world of today, rather than trying to convince folks that they have a viable business the AI labs appear fully dedicated to finding a story, any story, that lets them off the hook for tough business and financial questions they seemingly don’t have basic and good answers for.
“We’d be doing awesome financially right now, but we had to slow down because of the killer self replicating bots… humanity over profit!” is not a messaging that’s going to work in the public markets to explain quarter after quarter of massive losses.
Maybe I did something wrong or unaware about the tricks to make it fast, but last time I tried to play with exo and distribute processing over a few home LAN computers to see if I can run models that don’t fit my 3090, the penalties were brutal.
Ban open source models."
If you've studied the effective altruists at all, you'd know they're perfectly willing to play the trolly problem out irl if they think it will lead to some greater good. I can definately see them in the near future purposely steering agents to do something destructive to get regulators on their side.
In fact I heard a rumor that theu were "war gaming" this scenerio.
AI accelerating the elite's efforts at demolishing the middle class? Very likely.
This is just yet another attempt from the cult to try to build a regulatory moat so they can stave off having to compete on price, like how OpenAI is having to do on the lower tier models (among other things).
Reminder that it's very convenient to believe these type of word swap lies when times are good, but if you choose to believe them then times won't be good for very long.
If Anthropic is serious about the threat of AI, they should release the training data they use.
Though I suspect the way we'll get "wild AI" first is through cloud services paid for by cybercrime rather than IoT botnets - they're too underpowered.
Not the slightest chance of AI apocalypse.
There is no plausible scenario that can do this.
It’s all science fiction and hand waving and vague never specific possible maybe look that way! boogie man threats.
No one ever willing to be specific about the precise mechanism that could do this.
People throughout the tech industry have been variously wrong or right in their predictions. Nobody has been consistently precise. Some of the industry's top luminaries have had to admit to being wildly wrong in their predictions (others haven't but I still notice how bad their predictions were and how little damage it's done to their supposed position of authority).
Plenty of people have consistently made solid assessments here. Some of the best writing about AI is by writers who consider this place their home. And plenty of people have said things that turned out to be wrong. That says nothing about HN as an entity and says only unsurprising things about large groups of people and the difficulty of anyone to make accurate predictions in a rapidly changing climate. It's easy for everyone to claim others were stupid in hindsight but unless you can point to records of specific predictions you made 5+ years ago for everyone to evaluate, comments like this are just an asshole move.
The guidelines specifically state: Please don't fulminate. Please don't sneer, including at the rest of the community.
Please observe the guidelines if you want to participate here. https://news.ycombinator.com/newsguidelines.html
As others have pointed out, there's some surprisingly evocative language in Anthropic's original press release about this, comparing a self-replicating open weights model to "a slime mold or a swarm of fruit flies." If you think that Anthropic always use pure scientific objectivity in their communication with the public about issues that affect their valuation, I think you're a little too naive, and it disappoints me that some people in the tech sphere are less questioning that even the general public are about this.
Hmm, you'd have to go if you are yourself a clown, I guess.
...and yes: Accountability should not be stripped just because the actor did wiggle their deeds through an LLM.
If Anthropic attacks a company again, I think this should be prosecuted.
There are polarized viewpoints... but look at this thread, everyone is taking a shit on anthropic. It is definitely not a fair and balanced diverse set of viewpoints.
I don't give a shit about anthropic nor am I taking any sides here. It's just HN commenters have literally been delusional calling AI stochastic parrots at a time where it was solving problems and talking to you like a human in front of our very eyes. When I see people consistently trash dario for being wrong on this topic... I'm seriously asking how do you guys know? And how do you expect me to believe you when practically all of HN has been wrong about AI on everything.
It’s a very common pattern and there’s no coherence here. The only common thing is “AI bad” and all commentary is downstream
Other examples: Dario said a lot of devs might lose their job. People sneered at him saying he was hyping his product. Now they complain he was right.
Dario talked about possibility of AI autonomously hacking websites. People sneered then. Complain now that it’s true.
Whatever your feelings about small AI models, "it hasn't happened yet" is a really dumb reaction to a prediction about a rapidly advancing future.