I personally believe that it'll be normalized that mid-to-large size companies will build their own stack of compute for local ai model inference (some already have).
As we'll see more companies stacking hardware that isn't utilized to 100% at all times, do you believe that we'll see cross-company agreements to share this compute with other companies, for a given fee (i.e. the old school concept of time-sharing.)?
https://www.gpunex.com/blog/sell-computing-power-best-platfo...
Aren't Virtual Private Servers a form of time-sharing ?
VPS well, yes - but also no, you don't rent your self in on Costcos servers. The "cloud era" arose because the cost of VPS became so much cheaper than having all servers stacked in-house, and the fact that they provide easier maintenance and off-loads various data security risks that a regular company would have difficulties to comply with. I.e. higher up-time, lower latency for customers, less data security risks, less maintenance, lower cost.
But what we see with GPUs and AI driven compute, it can effectively be done locally, at a cheaper cost depending on the quantity of compute, requires little maintenance, the output can be offloaded on a cloud hosted VPS, the customer interface can still be routed through a cloud hosted VPS, but from a data security perspective you may deem it to be less risky to run local models on local compute.
I guess you could see XAI's (XSI's?) deal with Anthropic for compute as a sort of time-sharing, but I expect us to see these kind of deals being brokered between "non-tech" companies - unless token costs fall dramatically and unless it'll be truly ensured that all token processing is conducted in fully encrypted and closed of environments that are not used for training at all.
We're currently seeing a massive compute bottleneck - GPUs are expensive to buy and people/corps want to push as many tokens as possible into various models. Hence - Timesharing: A Solution to Computer Bottlenecks https://www.youtube.com/watch?v=Q07PhW5sCEk
Perhaps OpenAI's biggest moat in the future will not be that they provide an AI model, but rather that they facilitate a time-share structure for its clients. They will still be able to provide the most cutting edge model, but companies themselves will be able to run local inference on their own compute.