So what's the theory that goes with this about why cnbc are reporting that openai are walking because they want newer nvidia hardware? CNBC are clueleess? People at openai are lying to cnbc? cnbc are fabricating stories while drunk?

There has to be some theory to explain the story to be consistent with this comment.

Plenty of enterprise server hardware (racks, servers, RAM, disks) does have an active secondhand market after 3-5 years of use, but I think GPUs are too specialized for it to be viable. I doubt anyone has the setup to run a H200 in their home rig.

I also don't think companies are going to have mandatory replacement cycles for GPU hardware the same way they do for everything else, because:

1. It is an order or magnitude (or more) more expensive.

2. It isn't clear whether Moore's law will apply to the AI GPU space the same way it has for everything else.

Unless Nvidia can launch a new chip every 2-3 years with massively improved performance-per-watt at a lower price no one is going to rush to recycle the old one.

Depending on the elemental composition, it could definitely be worthwhile to recycle wherever scale is practical. For giant datacenters and companies using hundreds of thousands or millions of gpus, that adds up to a lot of gold and other valuable elements.

In order to take advantage of that, someone needs to be positioned to process all that material economically, and to make the logistics achievable by the big players. If it costs Facebook $10million to store and transport phased out gpus vs just sending them to a landfil, they're not going to do it. If they get $100k for recycling - probably not going to do it. If they pocket $5 million, they will definitely contract that out, especially if it costs $50 million to build out the infrastructure to handle it.

Probably a good company idea - transport, disposal, refurbishment of out of cycle GPUs and datacenter assets, creating a massive recycling pipeline for recapturing all the valuable elements is a pretty good niche.

It's likely the GPU boards are designed for water cooled data center racks and might not fit in a regular PC case. It's also possible the PCB the GPU's are mounted to might not be standard PCIe cards that fit into an ATX case.

I bought a used NEC SX Aurora TSUBASA (PCIe x16 board that looks like a GPU board) and realized it has no fans. The server case it is designed to fit into is pressurized by fans forcing air through eight cards on a special 4 + 4 slot motherboard. I have to stack and mount three 40mm fans on the back.

They are build to physically last 5-7 years in 24/7 datacenter use, but they have effective lifetime just 3-4 years, then their value has deprecated and electricity and infrastructure cost dominates. Meta did a benchmark where 9% of the chips failed every year, 'infant mortality' is much higher in the first 3 months of use.

Last I checked AWS is still offering g4dn instances that run on NVIDIA T4 GPUs, which were first released in 2018. I think most people underestimate how long superscalers can keep these things running profitably after they depreciate, and you probably don’t want anything they throw away.

My last employer is still running a bunch of otherwise discontinued g3 instances with 2015 era GPUs.

Bin!!

Why would them sell it cheaper to the 2nd market??

It will hurt the sales of new ones. This is the way even with food, let alone technology. Don't expect to buy cheaper 2nd GPU any century soon.

It seems like GPUs with a high utilization rate (60%+) degrade after 1-3 years: https://www.tomshardware.com/pc-components/gpus/datacenter-g...

Would be interested to know if others have takes on this.

I've written about this elsewhere but I predict there will be a significant secondary market for repurposing parts of datacenter GPUs (for example, RAM chips) by desoldering them and soldering them onto new PCBs that fit PC/consumer use cases.

You send them back to Nvidia or a third party e-waste recycler at end of life. Sometimes they're resold and reused, but my understanding is that most are eventually processed for materials.

Data centres are actually prohibited from using consumer level GPUs via license restrictions. The GPUs they use are largely SXM (server connector) and if you did somehow get one of the PCIe variants (with enormous power and cooling needs) most don't even support gaming APIs.

too bad stonk is down 23% this year. i think they are doing it wrong

The actions of oracle lately seem extremely misaligned to maximize stonks - it's extremely political, more than is necessary to merely keep in the good graces of the current administration.

TFA says nothing about taxpayer money - this is about Oracle taking on debt...

what taxpayer money?

While Trump is in power, the bail out is a sure thing.