Promethevs: Full DeepSeek-v4.1-Flash-1017, locally, on 12GB of VRAM(github.com)
13 pointsby ar-nelson7 hours ago1 comment
  • rvz6 hours ago
    If you were wondering:

    > Up to 7.4 tokens/s.

    It's not really "Flash" if it is 7 tokens per second.

    Also don't bother with any GPU lower than an RTX 5080 or even a DGX Spark (it is a scam) if you are seriously considering running frontier local models.

    • pu_pe4 hours ago
      It's apparently running at ~11 tok/s on a 4090: https://github.com/Magnvs7/Promethevs/issues/7

      Strata runs considerably faster than that on similar hardware, though this one doesn't seem to be a fork.

    • Ancapistani5 hours ago
      Oh, I don't know - it could be useful for long-running and/or background tasks in some cases.

      The problem is that this particular model is so cheap, compared to others. Optimizing to host it locally with relatively poor performance just doesn't make sense when yesterday I burned ~180M tokens for $2.20.

      • cassianolealan hour ago
        Surely local hosting is not so much about cost savings as it is about data ownership and risk management. e.g. can't leak a secret if you're not calling out.
    • a3w6 hours ago
      Strata for running from HDD seems really popular, to get big models running.

      Is that faster?