Any given AI to date has had weaknesses where some humans do better.

If this remains true indefinitely, humans can always fill that niche and be rich.

If it doesn't, you're wildly underestimating how badly it can go wrong for humans, because "things humanity collectively can do which robots controlled purely by AI currently cannot do" currently includes "making compute and robots for the AI".

If AI gets good enough to close this loop, the "minimally necessary human buffer" is "zero"... and that includes the would-be-rich.

Yes, this is the plan. The top 10% already hold most of the wealth and they are gaining an increasing share, with the top 0.1% gaining the largest share in recent history.

https://en.wikipedia.org/wiki/Wealth_inequality_in_the_Unite...

The end state of capitalism is an ever growing Gini coefficient (disclaimer: I say that without having done any research). Corruption will be rampant and individuals will have the power of entire nation states.

But despite that, the middle class may still prosper. In fact the rich will ensure it, for without a prosperous middle class there will be discontent and uprisings against their oppressors.

So surprisingly, despite a runaway concentration of wealth, capitalism is probably surprisingly stable. As long as technological advances continue, that is. As long as power can be grabbed by other means than zero-sum games such as territorial expansion and resource grabbing. Without that, war and conflict returns en masse.

So with that perspective in mind, I'm happy that AI happened. Without AI, growth was stagnating. With AI we get a few more decades of peace hopefully giving us the time to become a multi-planetary species, save the climate, and solve longevity (longevity probably being important for the human race in order to successfully populate the galaxy, since it otherwise becomes hard to justify and maintain the multi-generational bets of traveling to other solar systems)

So I'm not so concerned about AI alignment as I am with the human alignment of our future oligarchs.

Maybe