Carr was saying, in a hand-wavy way, that we'll see a step-change in technological advancement when AI innovation in physics bears fruit, and that other applications of AI are less likely to be as transformative.
I understand that we live in peace times and that there might exist authoritarian regimes in the future that could misuse government powers, is that what it comes down to? We should keep governments weak so that they cannot hurt us in case they go rogue? Reducing crime and disease is a goal that needs to be resigned to in order to avoid paying the apparently worse price of government abuse?
I'd rather we join forces by agreeing that the ideal is an all-seeing system that doesn't abuse their powers, can't we have the cake and eat it too? Or in the inverse, I'd rather we join forces on the basis that governments that abuse their powers are an enemy of crime and disease reduction, because they give a legitimate reason to destroy these powers that could be used for good.
There's a lot of talk about mission-driven companies to avoid future corruption. I'd be more than willing to fight for some mission driven protections that would avoid a company subverting the original mission of a company. But it's really hard to define what that corruption would be and even harder to make something that would be resistant to the government that charters the company itself, I think the bottom line is that I have a base feeling that pathogens and criminals are a greater threat than police and government, which feels like a huge political divide that is hard to surmount, but hopefully there's a way to keep the conversation open rather than pretend half of the world is the enemy of each other.
Going forward computer systems are going to have a lot of information on everyone - I don't think you can stop that. But the murdering people thing is what you want to try to stop. We could be doing better. Trump and Putin are not a great example - Putin kills a bunch of Ukrainians to take their land and Trump's main reaction is to shout at them to surrender the land as long as he gets some business deals out of it rather than shunning Putin. Ah well.