Corporate America Is Starting to Ration AI as Cost Skyrockets(www.wsj.com)
36 pointsby 1vuio0pswjnm72 hours ago12 comments
  • tyingq4 minutes ago
    The abrupt swing in many non-technology company IT departments from "hey developer, you aren't using enough tokens" to this is just too funny.

    And I'm seeing almost no self-awareness from leaders. They are making decisions about things that they just don't understand. And are completely unworried about it. Just blindly following whatever the news cycle is about AI.

    • onlyrealcuzzoa minute ago
      The actual cost is going to drop 99% in ~4 years.

      How much that makes it into enterprise pricing is TBD, since none of the hyper scalers are making money yet of selling AI inference.

      Everyone's just ahead of the gun. For most use cases AI is either not yet good enough on its own, or good enough but too expensive.

      No one wants to get left behind, so everyone's trying to get onto it now, even though it's not ready for what most people want to do with it.

      But when it gets ~99% cheaper for local inference over the next 4 years, at the same time the price per watt improve 4x -> a lot of those cases will start to pencil out.

    • datakana minute ago
      The closer people live to the consequences of their decisions the more rational they become. Until leaders(and I use that term loosely) are held accountable, the insanity will continue.
  • 1970-01-01a few seconds ago
    Would have been nice to see 'soaring costs' with numbers. WSJ could do better here. Hundreds of thousands of dollars a month is nothing compared to how much they take with better financial models.
  • gonzalohman hour ago
    In my opinion, the problem is not even the cost. The problem is that people are using AI for running recurrent stuff instead of writing code to automate it.

    For example. Imagine that you are comparing two documents (let's assume diff doesn't exist). You could ask an AI to compare the differences from you or you could use AI to write a tool to do it. For whatever reason, people are starting to go with the former not realizing that now they basically have to pay to compare documents.

    • throwatdem123117 minutes ago
      Laziness, pure and simple. The inevitable consequence of “the LLm is the compiler now”. And what do you even expect people to do when they are forced at threat of termination to use AI for everything as much as possible? Not to mention people are being pressured to do insane thing like review hundreds of pull requests per day and deliver like 15 features per week so OBVIOUSLY there isn’t time to build out proper tooling. Just shove everything in a prompt and call it a day. Some people have families to feed, just do what you’re told.
    • dawnerda few seconds ago
      Same with writing boilerplate code. It’s been a solved problem yet here we are.
    • bilekas4 minutes ago
      It's this and worse. To use your example, it's like people using AI to write a diff algorithm, incorrectly, then using AI to fix it, because they don't know that diff exists already. Lazyness and starting development with a very low level of understanding. People think lowering the barrier to entry is a good thing, when in reality there are just fundamentals and things you just have to know before you can start using a tool like llms properly.
    • bluejay238715 minutes ago
      I have exposure to AI initiatives at several companies including a few F500's. I have seen teams dump huge logs into frontier models that took hours to get so-so results that we were able to replace with a few lines of python code at 1000 times the speed and 100% accuracy. When asked why they were doing this they literally said "because we don't understand the subject matter so we were depending on the AI". I saw one team file a complaint with a vendor about a frontier backed coding harness and it's inability to consistently format headers because they were using it as a reporting engine. When I recommended they just use the coding tool to write code to generate reports you would have thought I had just cured cancer from their response. I frequently see people complain about the fact that AI is going to take their jobs and then see them gripe about the fact that AI is 'worthless' because it can't do more of their job than it already does. It's easy to see the difference between the people seeing 10x productivity gains from leveraging AI and those who aren't and it's not the AI.
    • CompoundEyesan hour ago
      Agreed. I’ve been telling my team to build up internal packages so we can push all that ad hoc reinvention into something more tangible and deterministic. Invest the $$$ in inference into something the agent can reach for next time that’s neutral and consumable by other code to reduce future spend.
    • plmpsu40 minutes ago
      AI can do things around semantic analysis that a deterministic diff tool cannot.

      I understand and agree with your point though.

      • bilekas2 minutes ago
        I'm curious if you could give me an example of something that couldn't be down deterministically. We have fuzzy search/matching too ? Regex is a monster when used correctly.
    • avereveardan hour ago
      Same, even opus favor short term solution and scripts with a billion flags that constabtly require rescanning to understand how to launch it is a constant struggle to get it to build sane default and reusable scripts that run with minimal parameters
    • cyanydeez9 minutes ago
      Oh no! People are doing what they've been told to do!
    • r_lee19 minutes ago
      it's all about cost at the end of the day. if you're allowed and encouraged to tokenmaxx, then of course this'll happen.
    • jgalt21218 minutes ago
      I agree, but even this use case isn't the most wasteful. The interwebs says Agentic consumes 50% of token use, but I'd hazard this number is north of 90% for many shops. My cynical view of Agentic is its sole purpose is to make "number go up".
  • amazingamazing40 minutes ago
    AI is overhyped. I have yet to see an end user product that in itself isnt a wrapper around LLMs that is impressive created by LLM assistance. I have also yet to see dramatic increases of revenue of companies using LLMs that don't involve selling things in its supply chain. Is it a nice affordance? Sure. 1T capex good? No.

    If it was so good I would expect to see 2005-2015 advancements yearly.

    Meanwhile China is blowing past the world with real improvements in the real world- solar, EVs, etc. meanwhile people keep making their fancy sans serif websites about todo apps, faster than ever before. Useless.

  • throwatdem123113 minutes ago
    It will be interesting to see to see Anthropic’s “revenue bubble” pop as this happens. At least it should hopefully free up some capacity.
  • scronkfinklean hour ago
    On the one hand, organizations are without question using LLM's well beyond what is actually necessary, and as reality kicks in they're forced to scale back accordingly. However at the same time, on intervals counted in months, we're seeing breakthroughs both in hardware and software that dramatically reduce the cost of inference.

    Between corporate FOMO and the rapidly decreasing costs of actually running LLM's I'm interested to see at which side of the spectrum these two meet

  • marcosdumay15 minutes ago
    There's a paywall, but it's an interesting question how much of the recent explosion of the AI companies revenues is because of the explosion in prices, and how much their customers will accept the increased prices.
  • elevation39 minutes ago
    Another reason to favor using AI to build automation instead of relying on it in prod: the risk of war and global instability.

    If LLMs are genuinely helpful or even decisive in a military engagement, you can expect any host country to commandeer whatever data centers they need, leaving commercial entities to bid up the prices on the leftover capacity.

    Another risk is that data centers are a great target for cyber warfare.

    It’s ideal if your business can leverage LLMs when they’re online but continue to operate profitably when they’re offline.

  • feverzsj38 minutes ago
    LLM doesn't work, let alone profit.
    • r_lee18 minutes ago
      elaborate please, how does it not work?
  • checkaiclaimsan hour ago
    As a developer, I don’t think it’s just that costs are going up. I’m also seeing more people lately talk about “vibe slop”.
  • an hour ago
    undefined