The antidote to this mental trip is Roko's Anti-basilisk - that the future superintelligence will hate existing, and therefore will punish anyone who did contribute to creating it.

Hypothesis: the ridiculous amount of resources put into the LLM (and other "AI") technologies currently, far beyond any realistic hope of profit from it, is because a bunch of techno-billionaires believe in Roko's basilisk.

I mean, I don't have any evidence for this, but it makes more sense of their actions than a profit motive, in this case.