58 pointsby johnbarron6 hours ago6 comments
  • reilly30002 hours ago
    I think this whole premise is ignoring the point that most decently sized companies want to and eventually will be running their own LLM workloads. Currently use cases are limited by scope and imagination, and predominantly focused on cost efficiency. Once a use case becomes a top line revenue driver budgets will become essentially only limited by ROI. There are some inference workloads that are unacceptable to send to the frontier labs for privacy reasons. Most of it will come from firms who want to productionalize their own fine-tunes. In any case, the market for inference is less than 1% of what it will be in 5-10 years. This whole notion that GPUs will be sitting idle en masse is ridiculous. People will just start running GPU databases if it becomes cost efficient.
    • reticulates2 hours ago
      but this is just an absolute fantasy, what, exactly, will this compute be doing? Our lives are already deeply entwined with technology and use barely any compute. How could our lives become 100x more dependent on compute?

      You can be thrilled by this exciting technology and all the possibilities it brings without thinking it is going to require this huge capital investment in GPUs. You could radically change billions of lives with a few dozen GPUs.

      • b1gOhbuddy22 minutes ago
        Ditch the servers.

        Do and sync over client to client.

        Keep data local again.

        Only use cloud for backups of local client encrypted blobs of vectors:data

        If you get rid of a lot of the suspect semantics hallucinated up over decades of software development it's not hard to see the geometry of an electronic snowflake. All the language just obfuscates the elegance. Crude meat suit grunts and clicks.

        Streamline it all to management of geometric states and access control and put the semantics on the presentation layer. What if we don't need python and go and ruby anymore? Made sense in a pre-gpu everywhere reality. Could just be high school stats classes to generate sets of values. Let go of the obscure linguistic chants.

        The data centers are just to serve surveillance purposes. Obfuscated behind politically correct memes of creating jobs.

        Chip away at the monolith and atomize the topology

      • dgellow2 hours ago
        Push everybody to buy hardware they don’t need to constantly run agents for pretty basic tasks, such as processing your daily emails and sending you little summaries notifications. So, the most inefficient software tools ever produced, but that keeps the whole industry alive. That’s pretty much the vision Jensen Huang is selling to ensure NVIDIA continues to grow
    • Reviving15142 hours ago
      TIL about GPU databases. I did a bit if research but only found this one: https://github.com/bakks/virginian

      What are the advantages of a GPU database?

      • yunnppan hour ago
        TIL too, but probably what GPUs are good at: optimizing for throughput while hiding latency. Probably better suited for OLAP than OLTP.
      • nr3784 minutes ago
        [dead]
    • SahAssar2 hours ago
      Is anyone really running GPU databases in prod at scale? Or are you assuming that the GPU compute from a crash will become so cheap that it makes this niche scale?
    • mikae12 hours ago
      > most decently sized companies want to and eventually will be running their own LLM workloads

      And probably some decently sized states too. Not only commercial actors are up to the job.

    • iwontberude2 hours ago
      [dead]
  • spaintechan hour ago
    You can like the character or not, but there is a trend I’m following ( heavily vested in NVIDIA, so tongue in cheek when I say this ) that might be highly align with Zitron. Looking at the moves from NVIDIA ( Groq )and AMD ( Taalas ) which are pure inference plays. I believe this shows that the impetus to train a better-bigger model might be coming to a level of maturity that might merit a serious threat to the frontier labs.

    For frontier labs, their fund-train-new model play might not be as effective, and a shift of spent of compute cost moving away from training to inference might be a tell-tell sign of the LLM as we know it plateauing out as scale is just not as effective. Open models might also be placing a major pressure on meeting then revenue targets need to sustain the model, lots of customer hosting their own inference to mitigate costs.

    If you only move the needle just slightly in the direction of inference, frontier labs will soon loose their alphas. Becoming just another SaaS for inference might not be as attractive unless you are Google/MSF ( IMHO ).

    Should this pan out, it could be a scenario where the NeoClouds could soon loose their biggest customers, so I tend to agree with that aspect of Zitron’s view.

    Thoughts?

  • jrflo2 hours ago
    I don't know much about this guy other than every time I see him on here he has an axe to grind about AI
    • beering2 hours ago
      You could start a new business selling grindstones to the axe grinders. Like selling pickaxes in a gold rush.
    • 2 hours ago
      undefined
  • vikramkr3 hours ago
    Ai revenue or datacenter/compute revenue? There's a lot of circular financing right now and there's a pretty obvious bubble for sure but I haven't been able to figure out what exactly this guy's argument is after seeing it a few times recently. Like yeah most ai datacenter spend is those two companies - that's pretty standard monopoly (or monopsony for the cloud providers) dynamics. If you're saying some major percentage of all ai related spending is openai and anthropic spending money on compute - well that's not exactly right because Google etc are also spending money on building datacenters (that's not ai spend in this definition? WTF is ai revenue exactly?) - and that's just pretty much indicating it's a frothy market with two unprofitable companies at the center with huge cogs? We knew that already.

    Feels like they want to make a clean headline grabbing argument about how "70% of all the spending is actually just these two companies" and are ending up with a really muddled headline that's just like yeah that's how monopoloies and duopolies work. When there's a lot more insidious circular complicated shenanigans going on that gets collapsed by this framing.

    • lokar2 hours ago
      The general concern (from him and others) is that the cloud providers are making extremely large capex commitments (borrowing, going cashflow negative) to satisfy demand from a small number of customers who may not be able to pay them.
      • re-thc2 hours ago
        > to satisfy demand from a small number of customers who may not be able to pay them

        Because a lot of isn't real demand, e.g. given away for free or very very cheap.

        • lokaran hour ago
          That’s really secondary, they don’t have the money and it’s not clear they will.
    • johnbarron3 hours ago
      >> When there's a lot more insidious circular complicated shenanigans going on that gets collapsed by this framing.

      All that is discussed in the video, plus those distinctions. And most important, that AI revenue would not exist...if OpenAI and Anthropic would not be funded, by the same Amazon, Google and Microsoft they are buying it from!

      We now have several voices saying the same:

      "Aswath Damodaran: Big Tech Has No Idea How AI Pays Off" - https://news.ycombinator.com/item?id=49229981

      "Why Wall Street Is Ignoring Big Tech's Debt" - https://news.ycombinator.com/item?id=49230630

    • doctorpangloss2 hours ago
      One interpretation is that so called profitability is one 15 minute phone call between OpenAI and Anthropic to increase prices. If you think that antitrust is the reason that this won't happen, you've stated a speculation. Not a certainty.
      • dgellow2 hours ago
        They are pressured by open models to reduce prices, not increase
        • doctorpanglossan hour ago
          Why do people continue to use them despite the existence of so many open weights models? This is "priced in." Short of stealing the checkpoints straight from the servers, it's unlikely to change.
          • WhrRTheBaboons18 minutes ago
            >Why do people continue to use them despite the existence of so many open weights models?

            Because of the low prices lol.

      • bvghnnxeyjbx2 hours ago
        there isn't enough demand to raise prices. OpenAI announced price CUTS recently and they are hilariously unprofitable already. Why would they be cutting prices if they have the ability to raise them? outside of this website and the ownership class everyone hates AI
        • Ekaros2 hours ago
          And I am not even sure ownership class have any real understanding or opinion on AI. Just that it has been sold to them as something that will save wages and make money... If it does not or the narrative turns they will abandon it like any other idea they have been sold.
  • scotty793 hours ago
    Wasn't it always seen as a "winner takes all" business?
    • sethops13 hours ago
      There is no "all", that's the point. There is no pot of gold at the end of the AI rainbow.
      • dgellow2 hours ago
        There is some gold, but it’s a fairly small amount compared to the ATM discussed by AI companies (basically software generation, software security, audio transcripts, translation, image processing, image generation, search, etc). So it’s not like there won’t be an AI industry in the future but it won’t be absolutely everywhere the way the AI boosters are projecting. Inference will be a low margin industry, training will be capex constrained and likely low margin too. And some services on top will have decent margins. But nothing like the datacenters full of PHDs Amodei and Altman are dreaming about.

        Just like any other technology

      • TZubiri2 hours ago
        There's a lot of money changing hands, at least that's a spoil isn't it? And if the music stops, someone is going to end up holding it.
        • grey-area2 hours ago
          Well a lot of money is being spent, but is a lot of money being made?

          With circular deals it’s hard to tell.

          • dgellow2 hours ago
            NVIDIA, Micron, SK Hynix, and other hardware manufacturers, and also hyperscalers are the ones making the hard cash (unless you’re Oracle, that company is more than fucked)
          • scotty79an hour ago
            When somebody's spending someone else is making money.
            • grey-areaan hour ago
              With circular deals it’s hard to tell.
            • harpiaharpyjaan hour ago
              Except when it's that same someone else's money being spent.
    • amelius2 hours ago
      If there was one winner then that would make a lot of people very angry.
    • 271832 hours ago
      > Wasn't it always seen as a "winner takes all" business?

      No. It was always a pyramid scheme marketed as a "winner takes all" business. But there's no winning. The only winners are those who cash out before it collapses. This is literally Web3 2.0.

  • ElProlactin2 hours ago
    "Because Goldman does this kind of nonsense."

    Hilarious. You can agree or disagree with Zitron but he really has no business talking about Goldman Sachs. His posts are littered with evidence he has no ability to perform the type of financial analysis he thinks he does.

    • dgellow2 hours ago
      Mind sharing more details?
      • ElProlactin2 hours ago
        Sure. Here's an example:

        https://www.wheresyoured.at/exclusive-openai-financials/

        Zitron wrote:

        > Additional factors – including interest income and interest expense – left it with a net loss of $8.84 billion. It then marked $3.74 billion of losses as “net loss attributable to noncontrolling members capital,” leaving the net loss attributable to the company as $5.09 billion.

        > It’s unclear what this means, nor how OpenAI reconciled the removal of $3.74 billion in costs. I will not speculate further.

        It is very clear what this means, and no speculation is required if you understand basic consolidation accounting, which you would expect someone in his position to understand.

        It's not rocket science: when you have a parent company with entities it doesn't wholly own, the slice of losses belonging to the other equity holders is split out as "noncontrolling interests." Nothing is removed or hidden; the total loss is unchanged, it's just allocated to reflect that the parent company doesn't own the whole. Framing it as OpenAI removing costs implied something sketchy and requiring speculation where there's only routine GAAP accounting.

        But it's even worse than this. So many of Ed's claims conflate the foundational R&D and capital expenditures these companies are incurring with the unit economics of their businesses. He seems woefully unable to understand that you could sped gobs of money on the former and still have positive gross margins that scale over time with the latter.

        • Driver4732an hour ago
          But how does that detract from the overall point that Open AI is losing billions of dollars? If there's an insatiable demand for AI compute, where's the profit?
          • nr37818 minutes ago
            High growth companies often have significant negative cashflow during the early high growth era, followed by positive cashflow in the years later down the line.

            This phenomenon is known as the J-curve[1], and Uber is a good example of how this can turn out absolutely fine. To some extent, the entire Venture Capital industry exists to finance precisely this dynamic!

            Nb. I'm not suggesting OpenAI is fairly valued, or that they will definitely become profitable, but "OpenAI is losing billions of dollars" doesn't really mean anything in and of itself.

            [1] https://www.uark.vc/blog/breaking-down-the-j-curve-the-journ... (many other similar such articles exist)

          • ElProlactinan hour ago
            First and foremost, it's about competent reporting. You can think a company is doomed and still expect people to report on it accurately.

            Zitron continually presents things in ways that create a hyperbolic narrative. Turning routine consolidation accounting into an unexplained mystery hinting at some sort of fraud is the perfect example of that. He does it so often and in such a way that I truly believe he just doesn't understand accounting.

            It doesn't take a rocket scientist to understand that OpenAI is losing money. But the question ("where's the profit?") assumes that profit is the thing being optimized for. It isn't.

            There are basically three buckets here: cost of serving a query, cost of training and capex for capacity.

            The first one is unit economics. The other two are bets on the future that get expensed against present revenue. A company can serve every query at a healthy gross margin (OpenAI has improved margins considerably) and still have a $9 billion loss because it spent $12 billion training a model that generates $0 this year.

            Zitron constantly blends everything into a pithy "they lose money on everything" narrative, which just isn't accurate. The thing is that OpenAI could have a very different P&L if it chose to, say, stop training the next model.

            The problem, obviously, is that if you stop training the next model, the competition might eat you. So right now you have a situation where the the frontier model you spent $12 billion on depreciates in about 18 months, the GPUs depreciate on a schedule nobody agrees on, and you seemingly can't stop the cycle without risking your position in the market.

            This is the legitimate bear case, but the problem with Zitron is that he doesn't make it using an argument that is coherent and honest as far as the accounting is concerned. And the accounting is everything.

        • grey-area40 minutes ago
          Can you explain which partly owned subsidiaries lost $3.74 billion and why it is legitimate to exclude that from their losses in a non-handwavy fashion? Your condemnation leaves me none the wiser and this does matter.

          This can of course be used to distort the financial picture and this is a significant amount, almost 50% of losses. Is this from the ‘non-profit’ which used to be OpenAI or something else?

          Smells like creative accounting to me and the CEO was accused by his board of dishonesty.

          • ElProlactin35 minutes ago
            I explained this is a different comment. This is basic consolidation accounting per ASC 810-10-45.

            https://dart.deloitte.com/USDART/home/codification/broad-tra...

            I get that not everyone is an accountant or has had to become educated in accounting matters as part of their work, but you really shouldn't say "smells like creative accounting to me" if you don't have a basic understanding of the subject.

            This is like the least interesting thing about OpenAI's financials, and Zitron framing it as some sort of mystery hinting at fraud is one of the least effective ways to make a point given that it's absolutely a nothingburger.

            No loss is disappearing or being hidden. This is by-the-book consolidation accounting.

        • beepbooptheoryan hour ago
          Maybe I am little confused here but what you just described here doesn't sound great either? What smaller entities who OpenAI is parent to lost 3.74 billion dollars?

          Or maybe just can you point to some primary sources about this? I am not too bright about this stuff.. I guess I always thought it was usually about having more money than when you started? Or at least about having a story of how you will have more money? Is that not right?

          • ElProlactin40 minutes ago
            You can look up ASC 810-10-45.

            OpenAI isn't a single company. I haven't followed all the details with its structure change/recapitalization, but it's (I believe) a parent sitting on top of an LLC that outside investors like Microsoft hold a large minority stake in.

            The rules say that the parent has to report 100% of the LLC's revenue and expenses as if it owned everything and then, at the end, back out the share of the loss that economically belongs to the minority holders.

            So $8.84 billion is the whole loss, $3.74 billion is approximately the outside members' proportional share of it, and $5.09 billion is what's left for the parent. Nothing disappeared or was hidden. It's one number presented two ways because two sets of people own it.

      • npilkan hour ago
        One example - he often compares current revenues to capex being spent on future capacity to claim that AI companies aren't profitable. (See this post for example: https://www.wheresyoured.at/am-i-meant-to-be-impressed/ .)

        But this ignores that the capex spent to build more capacity is expected to generate additional future revenue. You don't need to recoup your capex immediately. A better approach would be to amortize the capex and compare revenues to that.

        Clearly he assumes revenue won't increase enough to recoup this level of capex (and it's very possible it won't) but IMO it's either a miscalculation of how the financing works or a deliberately misleading framing to compare current small revenues to a big scary capex number.

        I'm sure the above is simplified by the way, but I am confident that people who work at Goldman understand the relevant details extremely well.