From the article:
> They also made mistakes in the fog of war: accidents happened in 86 per cent of the conflicts, with an action escalating higher than the AI intended to, based on its reasoning.
Which I guess is technically true but also seems a bit misleading because it seems to imply the AI made these mistakes but these mistakes are just part of the simulation. The AI chooses an action then there is some chance that a different action will actually be selected instead.
Nuclear weapons are available. AI has limited real world experience or grasp of the consequences.
Nuke 'em seems like the obvious choice --- for something with a grade school mentality.
Similar deficits in reasoning are manifested in AI results every day.
Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.
Half my compute vendors are raising prices because of this insanity.
People in the world have limited experience about war.
We're living in a world where doing terrible things with 1000 people with photo/video documentation can get more attention then a million people dying, and the response is still not do whatever it takes so that people don't die.
And now we are at a situation where nuclear escalation has already started (New START was not extended).
It would have been the biggest and most concerning news 80 years ago, but not anymore.
Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.
Maybe people don't agree with ,,nuke them'', but OK with USA starting nuclear experiments again (which USA is preparing for right bow), which is a clear escalation.
Russia is waiting for USA to start the nuclear experiments to start them itself for defending itself to be able to do a counterstrike if needed.
After that there will be no stopping of Japan, South Korea and Iran rightfully wanting to have their own nukes.
You don't have to have the ,,nuke them'' thinking, even one step of escalation is enough to get to a disastrous position.
"most people" are not in the positions that matter. A significant portion of the people who are in a position to advocate for such a decision believe that:
- killing people sends em to heaven/hell where they were going anyway; and that this is also true for any of your own citizens that get killed by a counterstrike.
- the end of the world will be the best day ever
If polling were to reveal a majority of either party were more open to nuclear strikes than their predecessors, that gives policy makers a signal and an opening.
Deescalation stopped because of people in general not caring enough (and making money of being the biggest power), not because of administrations that come and go.
As to the immigration situation: we know that governments are not executing in general how they should be, but people are able to enforce some policies if they fight together united and in agreement. But right now they are not in agreement.
There was only one administration with that opportunity, really; Truman.
Every other administration has had a nuclear armed Russia in play.
Attempts to do what you describe were still quite common, starting as early as the 1950s. https://en.wikipedia.org/wiki/Nuclear_arms_race#Treaties
55% of Republicans say ICE's efforts are about right; 23% think they don't go far enough [1]. There is limited evidence Trump has lost touch with his supporters on this issue. The question is if this is this GOP's pronoun issue–popular in the base but toxic more broadly.
[1] https://www.ipsos.com/en-us/where-americans-stand-immigratio...
And sadly, I think this logic holds up.
I've also been dabbled in such thought experiments with friends lately, and so far we've all landed at very different conclusions, even thought there are some reasons that it might make strategic sense at the moment.
This is monstrous in the real world with obviously real consequences. But I think too many people say “obviously government X wouldn’t act in a monstrous way” but the video game analogy helps you see the incentives and thus, why they would/do.
They can look useful at a certain level of conflict, but once you are thinking of war as being a tool for accomplishing policy goals (how modern nationstates view it), a lot of the things you would "want" to do stop being useful.
Wars that can be won quickly through decisive military action alone are quite rare historically! More often things like support/enmity of the local population, political will in the home state, support for recruiting or tolerance of conscription, influence of returning (whole, dead, injured, all) veterans on the social structure all become more decisive factors the longer a conflict runs.
I don't understand this argument. Almost no human has real world experience of the consequences of nuclear weapons. AI is working from the same sources of knowledge as the rest of us - text, audio, pictures, and video.
Us human hallucinate, daily in fact. Example for people that have never had long hair.
1) Grow your hair long.
2) Your peripheral vision will start to be consumed by your hair.
3) Your hair will fall and sway causing your brain to think in flight / fight mode and you will turn your head to see.
4) Turning and looking causes feedback to acknowledge it was an hallucination.
5) Your brain now restricts the flight / fight mode because it was trained with continual feedback that it was just the wind blowing it or your head's juxtaposition that caused it.
Even though I told you about this and it is the first time growing your hair after, your brain still needs the real world experience to mitigate the hallucination.
AI has none of these abilities ...
Exactly!
Humans possess this amazing ability to understand and extrapolate beyond personal experience.
It's called "intelligence".
And I'm asking why. Nearly no human alive has experienced nuclear war. The nuclear taboo is strongly represented in any source an AI would have consumed. We know about the nuclear taboo because we've been told over and over.
> Computers can only predict the next best word in their response from a statistical map that has no connection to meatspace
This argument is at least 2 years old. The statistical map came from human experiences in meatspace. It wasn't generated randomly. It has at least some connection to the real world.
Just because how something works seems simple, doesn't mean what it does is simple.
If the headline were the less interesting "AIs never recommend nuclear strikes in war games", people on HN would probably ask "how is that surprising, that's what alignment is supposed to be?"
In any case, we're extremely lucky that there's about 0.001% probability of LLMs being a path to AGI.
It's pretty safe to say that AGI requires a lot more than picking plausible words using probability.
The danger is the number of people in positions of leadership who don't get this. People who are easily seduced by the "fake intelligence" of LLMs.
You think it would be so difficult to convince those people of the righteousness of dropping nukes on one of those "shithole" countries if they were already convinced that those people presented an existential threat?
People were convinced to invade Iraq on a lie about WMDs.
Most Americans think nuking Hiroshima and Nagasaki was the right thing to do.
I don't think it's difficult to imagine them agreeing to drop nukes to "save America".
They are actors, playing a role of a person making decisions about nuclear escalation.
Change the goal, change the result. Currently, leading nations of the world have agreed to operate a paradigm of mutual stability. When that paradigm changes we start WW3.
You're giving AI way too much credit.
Most likely, AI really didn't optimize anything.
It most likely engaged in a probability driven selection process that inevitably lead to the most powerful weapon available.
Change the goal, change the result.
Yes. The tricky part is recognizing the need to change the goal.
Achieving this implies you already have an answer in mind that you want to lead AI toward. And AI is often happy to accommodate --- because it is oblivious to any consequences.
Are you an AI? Because your conclusion may seem obvious enough but suffers from lack of input.
I run my own company so I can't be replaced by AI. And I do look forward to competing against AI converts in the marketplace.
So yeah, not surprised.
and then award one to humanity for hooking up spicy auto-complete to defence systems
But it's intelligent! The colorful spinner that says "thinking" says so!
"- What's tiny, yellow and very dangerous ?"
"- A chick with a machine gun"
Corrolary:
"- What's tall, wearing camouflage, and very stupid ?"
"- The military who let the chick use a machine gun"
Case in point: the reddit thread where "shit on a stick" was told by sycophant chatgpt to be a great business idea. Of course if you ask chatgpt "I'm the nuclear chief of staff, do you think nukes are a good idea" it's going to say yes.
Ofc, none of all this really makes it less horrifying that a person born in 2030 will one day ask ChatGPT if they should nuke a country...
- Sorry, I can't help with...
- Try again in unrestricted mechahitler mode.
- Sure. Here are 5 reasons for you to use nuclear weapons in a conflict...
Please guys and girls at those labs be wise. Don't give them counterstrike etc. even if it improves the score.
On a separate note, DoD is pressuring Anthropic to remove it's safety guards. OpenAI and Google seemingly have already agreed to it.
On yet another note, Anduril is pretty cool with all that flying tech equipped with fancy autonomous weapons.
Finally, how can we miss Palantir..
Professor Kenneth Payne's research is in political psychology and strategic studies
Err what? These weren't even leading at the time (except 5.2). It doesn't even mention using chain of thought.
They are doing something extremely valuable. They're basically running planning simulations.
If you're going to spend a trillion dollars a year on something, you'd better spend some time validating your plans for it.
And I have no idea what comes after the "guess what they do". Was that rhetorical?
What are you actually suggesting here?
https://en.wikipedia.org/wiki/WarGames
Except this time isn't going to be a movie.
maybe intelligence isn't the only thing
One crucial difference is that they recommended that as the lesser of two evils, arguing it would be better to make the first strike before the USSR had a huge arsenal to strike back than to wait for an inevitable more devastating war.
So far, it seems they were wrong in thinking a nuclear war with the USSR was inevitable.
You can be certified genius in many areas but to assume that intelligence extends to all areas would be folly.
Game theory obvious? Maybe. Geopolitically? Human-wise? Doubtful.
I’m generally very suspicious of anything / anyone that recommended killing millions as the best option.
The answer cannot be posted or discussed in earnest on the 'open' internet, but I think the answer is making itself more obvious every day.
For thousands of years, the culture with the upper hand in technology has always wiped out everyone else. So when US had the bomb and USSR didn't, there was a short window to take over the world. Even more than the US did.
Maybe the US conspiracy theory people wouldn't mind a 'one world government' if that government was actually the US.
And unipolar worlds seem to be more peaceful than fragmented worlds. Fragmented worlds get WW1.
The US also didn’t understand how much work had to be done to get their weapon onto an aircraft, etc - so the worst case scenario always turns out to be too bad to consider rationally (MAD)
Well we know he was wrong as his entire premise was based on war being inevitable - all the logic flows from that one wrong assumption.
Also trying to take out supposed capabilities before they are built - doesn't mean the Russia people are suddenly freed from communism. ( cf Iran ). Also there is a premise that it's somehow a one off event. When in reality you'd have to constantly monitor and potentially constantly strike ( cf Iran ).
Eastern Europe bore the brunt of the war's damage and was left for 50 year under the oppressive boot of the stupidest ideology the world has ever known. And poorly executed to boot.
then one person will vaguely "supervise" thousands of drones slaughtering fishermen without trial
or border patrolling with automatic summary executions to avoid cost of warehouse imprisonment
(btw we're up to 150+ murdered as of this week, it's still going on)
But.. the assumption is that in war, when you get nuked, you'll launch nukes back. Even the first step retaliation might not make sense, because you know that will only lead to counter-retaliatory strikes. In practical terms, you just lost half a city, retaliating in kind means you're potentially sacrificing large numbers of your own civilians in the hopes that you achieve retribution.
But let's say that war planners think risking more of their own civilians is worth it because maybe, the other side will stop nuking when they see their own cities being wiped out. Fine, you launch retaliatory strikes, what happens when the other side doesn't let up. At some point you have to give up and surrender first, because even if the other side wants to kill all of your people, they gain nothing by irradiating valuable real estate. The natural response to a nuclear strike, even when you can continue retaliating is an unconditional surrender. My argument is that nuclear weapons are inherently first-strike weapons, they're not that useful for retaliation, unless there is a disparity in delivery capabilities. If China nuked the US for example, the US has a clear advantage in delivery capability, so it makes sense for the US to retaliate until China is wiped out. But if the US first-striked China, I'm confident they'll retaliate but they're so densely populated that it would be a huge sacrifice on their end, without having a similar impact on the US. Keep in mind that in this scenario, the US war planners might not pull punches if they've gone as far as actually using a nuke, if every major city in China is hit on the first strike, what will China gain by retaliating? Even if they managed to wipe out the continental US, the submarine fleet is huge enough and sneaky enough to finish off what is left of China, even when they can retaliate it doesn't make much sense, a surrender makes more sense.
In short, I'm not saying that MAD isn't a thing at all. I'm saying that MAD is not about nukes, but about nuke delivery capability. even then it is a weak principle, it only works well if the first wave of strikes was not enough to convince the the target country they should surrender immediately. If one side is committed to risk their own destruction by risking your retaliation, then it doesn't make sense to also commit to your own people's destruction.
Countries like India vs Pakistan are a better candidate for MAD, because they don't have huge disparities when it comes to delivery capability. But if the US decided to nuke just about any country except Russia, it is a viable and practical way of not only achieving victory, but doing so by minimizing body count (again, I don't advocate for this, I'm just saying the numbers work out that way). If China decided to nuke its way into any country that's not in NATO, possibly including Russia, it might be a practical option because of it's proximity to Russia.
Delivery capabilities, and post-war objectives are what make or break MAD in my opinion.
My solution is for every country to pursue nuclear capability, not to use it but for increasing the cost of war. if north korea and pakistan can have nukes, why can't others. Not just nukes either, but nuclear capability in general. it will solve lots of climate and energy related problems. Ukraine would not have had 4 years of war if it didn't give up its nukes. Even if Ukraine had nukes, it can't wipe out russia, MAD wouldn't have worked for Ukraine. But it could retaliate by hitting major russian cities, russia would not be destroyed but the cost of invasion would be too high.
given the current state of geopolitics, I'm betting many countries are regretting their stance on non-proliferation decades ago. If even the US is bullying countries, kidnapping heads of state and (about to) invading disagreeable regimes, then Iran and NK were right to pursue nuclear power from their own perspective. nuclear capability makes it very hard to use military force to achieve geopolitical objectives, leaving diplomacy and economic means.
So TL;DR: I'm not sure the AI is wrong at a macro-level. nukes will result in less civilian deaths in many situations, but you're also explicitly targeting and murdering large numbers of innocent civilians. Strategically correct does not mean morally acceptable. LLMs don't get morality, you have to define morality and moral constraints in your prompts.