Data center demand right now has much more to do with private credit's demand for new deals, since the binge they went on around the pandemic led to the accumulation of a lot of bad debt and depreciating assets. Data centers are a tool for extracting money from insurance trusts and pension funds to prop up private credit and create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
I'm building a side business and decided to go with open router because that's what AI is going to look like in the future. Just intelligence/$ everything else kind of not important.
Unless, they solve the issue of model output diversity and Claude becomes very different from ChatGPT
Sure, Claude and ChatGPT are electricity, but oil is not just gasoline and diesel. It's also responsible for the colors all over our world. Plastic comes from oil. So chatbots might be electricity but the broader field of AI seems more like oil to me.
AFAICT, this hasn't happened for any "cloud" compute company that has been in business for -say- at least five years. Unless your prediction is that roughly no one will use "cloud" "AI", I'm rating this prediction as "totally bogus". And if that is your prediction, then what you're saying is
Assuming that you're not doing anything that needs exotic system administrators, nearly all of the cost of computing when you own your own hardware is the power for that hardware.
240V * 10A = 2.4kW
2.4kW * 24H * 30 * 12 = 20,736 kWh
20,736kWh * 0.22 EUR/kWh = 4562 EUR/year
Note that I'm using roughly decade-old hardware prices and allegedly-current electricity prices. If we used hardware prices from two years ago -or worse, from today- the hardware would cost far more per year.
[0] <https://www.hetzner.com/colocation/>
[1] ...which I think is between two and three times longer than is standard to expect in the business world...
[2] ...it's very likely that I have! Corrections are definitely welcome.
[3] ...it's also possible that USians get gouged for server hardware and that it's like a quarter of the cost if you buy it in Finland. I expect that there's definitely a price difference, but I doubt that it's very large.
Most cloud software has had moats. AI does not. And AI is eating software, which means those historical margins are going away for a lot of cloud software,too. That's what the SAAS-pocalypse is about.
>Assuming that you're not doing anything that needs exotic system administrators
Remember, these are also going to be AI.
>Assuming I haven't fucked up my math, [2][3] the cost of the hardware here is more than three times the cost of power.
This is a fair point, but it just means the COGS is higher, not that the margin is any better. I stand by the prediction that AI margins will be very slim.
I'm very dubious we can have our cake and eat it of constant magnitude order improvements and also commodified utility pricing.
168 * 12 < 4562
168 * 12 < 4562
If we assume that Hetzner's spending cap *doesn't* apply...
If I were saying that, I would have said it. Unless "XXX max/mo." doesn't mean what it means in plain English, I _very_ strongly expect that it's very rare for colo customers to suck down all available power 24/7 and that Hetzner has a Stern Talk with customers who do but didn't disclose their intent to do so up front. I mean, you tell me... if you've ever lived in an apartment where your power and water was included in the rent and was metered with a whole-building meter (rather than per-unit meters), did you consume every amp and gallon that could be delivered to you, or did you use just a bit more than what was useful to you in the course of everyday life? [0]
I worked the math the way I did (even doubling or tripling the expected lifetime of the server hardware) to make the numbers as favorable for the "electricity is almost 100% of the cost of computing" side of the argument as they could reasonably be.
Are you saying that I fucked up my math?
[0] And if you're never lived in such an arrangement, ponder instead how you would behave if you did find yourself in one.
LLM are just now consuming data, ad networks have been gobbling on it for many years.
>create debt-driven artificial "revenue" for the hyperscalers and Nvidia.
what for? Cui bono?
AI companies, LLM makers, AI solution providers, and AI users should be inexcusably responsible for any and all harms caused by their misaligned automations across the chain. This alone will encourage self regulation and fix a broken industry that has too long prospered on fail fast and break things to be more considerate of their actions.
The second form of regulation should be on excessive energy consumption. The US Department of Energy needs to declare energy to be a vital human resource, and exorbitant uses for computing to be curtailed. Bidding on energy prices should be restricted to the excess markets where free market dictates price. And supply will reduce availability.
Long term, this will put the right kind of pressures and halting of demand. It will slow down investment in infrastructure and increase investment in research and development. It will prioritize efficient AI uses over low quality trial and error.
> Architect Financial Technologies, founded in 2023, offers futures in compute linked to the value of chips in America. The firm has also recently launched a platform where model-makers bid to provide “inference”—the useful output produced by an AI model in response to a prompt—rather than the underlying compute capacity.
I think the idea of selling inference is solid, but I’m not convinced Wall Street is going to be doing that. Superscalers are better positioned.
Inference will likely be priced by evals, specific for each workload.
Every other post on the News.YC frontpage has been about AI for the past few years. That means very little.
> YouTube has gotten a lot more entertainment rising above the slop.
Examples would be great. I'm looking on YouTube's "hot" playlists and haven't yet seen anything marked AI, although that of course doesn't rule out the use of AI.
The thing about AI entertainment is that it's far more targeted and niche, so you're unlikely to find huge central gatherings around one popular thing, but instead it's spread out over many smaller channels.
I think LLMs are more likely to end up as something boring we don't think about like cloud computing but I don't think comparisons with crypto etc are fair as it's a much more generalisable technology.
corruption: checked
regulatory capture: in progress
Ya, the new oil.