Training is 10-100x more compute than inference. The current rush is all training. When it tips over to mostly inference, a lot of that hardware is just going to sit there.
It's already happening. Cost of capital is increasing, Nvidia and SK Hynix going down. This will follow the patterns of a "real demand but overbuilt" bubble like railway back in the day.
I personally plant to buy GPUs in about 18 months.