1 pointby chrisjj5 hours ago1 comment
  • verdverm5 hours ago
    my hypothesis is they are using flash and quant'd models to keep up with demand in the hardware crunch

    You see the same pattern across open weight models and homelab setups, as we try to fit the models on devices while preserving capability and getting reasonable throughput

    • chrisjj4 hours ago
      Trying to preserve something one cannot reliably measure. I wish you luck!
      • verdverm3 hours ago
        > cannot reliably measure

        there are lots of things you can measure, designing tasks for evals is an art (which I'm trying to learn)

        • chrisjj3 hours ago
          Sure but I said reliably measure.