The Economics of Open-Weight Inference(data.ornn.com)
21 pointsby marinesebastian4 hours ago4 comments
  • jcmontx2 hours ago
    The problem, IMO, with open-weight models is that you accustom to the capabilities of frontier models too quickly; and downgrading to an open-weight "frontier minus 2" or "frontier minus 3" model is often painful, since they feel way less useful than their newer closed-weights counterpart. To be honest, I don't know any companies using OW models at a large scale for their operations (agents or chat assistants).
    • physicsguy13 minutes ago
      That is true if you're using them as chatbots but for something behind a product it's largely OK as long as it meets the requirements.
    • Silagian hour ago
      I think this is where Deepseek has nailed the mark; DS4.1 Flash is really, really fast, and really, really cheap. If you give it small, structured goals, it completes them crazy quick, at negligible cost. There's different vectors to differentiate along to stay in the conversation.

      I've taken to using them as micro-review subagents at development milestones, where a "frontier - 1" model like Opus or Sol launches 10-15 of them on small review tasks that each run for ~10 minutes. Costs about $1 per cycle, and they usually catch something Astra or Fable didn't. Then the orchestrator validates each claim before passing it back to the planning session so we can fold the findings in.

    • lowbloodsugar6 minutes ago
      I’ve found the opposite. Going from Opus 5 to Qwen3.8 flash has been a breath of fresh air.
  • augment_me2 hours ago
    I think an interesting point is that hardware as of today still has no utility value after its reported lifetime has elapsed, which prevents neolabs and smaller labs from getting older HW clusters as the banks are not willing to give out loans against them. There is no agreed upon pricing for "expired" A100 clusters or similar.

    This is clearly not true, and we are starting to see compute markets, but only for rental prices/H, not for the hardware itself. I feel like there is some artificial moat being built here to stimulate sales of new hardware, because an H100 at 1/16th the price will have comparable dollar/FLOP as Vera Rubin.

    • csmoak25 minutes ago
      https://www.stoaexchange.com/ is an exchange for hardware itself and has market data for prices over time.
    • linuxftw2 hours ago
      Depends on the workload. H100 will never have the network performance of Vera Rubin. There's also token per watt, newer systems will beat the older systems.
      • kurthr2 hours ago
        It's not clear how much of the latest chips have even made it on-line yet.

        The claims of many GW of installed training/inference have come under scrutiny lately. The first VeraRubins aren't even there yet, so it's all GB300 NVL72s as the peak performers and probably <<1GW of those so far. Even xAI Colossus is mostly H200s and B200s.

        Electricity costs are also a huge differentiator. When drawing 100kW the difference between >50cents and <10cents per kWh is pretty big! One is almost $0.5M and the other is less than $100k.

  • hungryhobbitan hour ago
    Dark grey text on a black background: it's like they are trying not to let anyone read their article!
  • simianwords2 hours ago
    Then why do people still believe that OpenAI any Anthropic have negative margins