Bears are gonna eat.
My thinking is there wouldn’t be downward pricing pressure on data center compute if utilization was actually this high - it would mean providers could jack up prices and walk with larger margins rather than burn money for acquisition, right?
Plus the market wouldn’t increase the price of borrowed money if utilization was that high; they’d see the same signal. But the cost to borrow money for datacenters has roughly doubled year on year (might just be for riskier ventures? Buildings? Is this aggregate?) - those two signals wouldn’t disagree, it’s nonsensical (I mean, right?)
Edit: it’s just “the buildings”, and it makes sense: maximal utility comes from installing compute in a building already setup to house compute (cooling, infra, power). I actually take this as a negative signal: we’re seeing downward pricing pressure already, AND new projects aren’t fully booked, AND money is now twice as expensive. Somebody’s going to lose their shirt unless OpenAI’s Omnissiah comes online and converts us all.
The real question IMO is building capex. Those depreciate over a longer timeline, but if demand were to drop in 5 years and we have an excess of datacenter space, that's going to hurt. We're approaching the physical limits of what's possible with silicon and fiber optics. Unless there are new architectures or break throughs in algorithms, the only way to make AI more productive today is to build larger clusters, or much longer training/RL runs.
- The factories don't stop working after 3 years. They can keep making GPUs.
- The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.
Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years
Once there is enough, or even a glut, it's over.
Is intelligence like for example food where if I get my daily calories I’m good? Or is there no limit to the demand for intelligence. If there’s more available, will we just keep using more? Forever? It can’t be limitless right?
But where’s the limit because right now we’re consuming every GPU hour we can.
But corporations? Government? Universities? Militaries? Biomedical research? I suspect the ceiling on their consumption of intelligence is much much higher than Tina’s.
Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.
Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.
Turns out running chips hard 24/7 causes them to burn out
Hopefully somebody with direct experience can correct me if I'm wrong...
4x3090s bought in 2021 also still work flawlessly (although they were permacapped at 250W to avoid fan noise).
His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.
And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in figuring out whether debt loads are likely to be sustainable or not. Investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.
It would be one thing if he once admitted that he was wrong and adapted his world view to account for new information, but he just doubles and triples down.
I doubt that many of the 6-year old A100s have been in use 24/7.
Likewise if models become commoditized, it's raising the value of the existing hardware (you'd expect the hardware owner to keep a bigger slice of the surplus when that's the scarce component).
For model improvements to be bad for the value of hardware, what you'd need is for demand for intelligence to be inelastic. But everything we've seen so far suggests that is not the case: the demand for intelligence (whether human or artificial) is highly elastic both in price and in quality.
until the other guys comes out with a slightly better model, then using yours wont be cheap enough.
No.
>Do you make a frontier model orchestrate it?
Also no.
>Are you running with llama.cpp?
Yes, on Arch Linux.
>Sorry for the barrage of questions
No worries.
My Qwen testing has been with Hermes trying to do mundane (to me) things that hosted models understandably refuse to do, even open-weight ones like Deepseek V4.1 Flash. Example: I tried to get it to find a torrent for $tvshow and add it to qbittorrent (connected to hermes via MCP already). It certainly tried and I think frontier models would have done better. My biggest issue was context space more than anything else. I had to drop down a quant level to increase it to 128k. On 64k it was constantly compacting.