Ask HN: Is it time to run the LLM engines on the CPU?
3 pointsby roschdal3 hours ago3 comments
  • dtagames3 hours ago
    You always could. It's just slower and less efficient because most of what a model query needs to do is matrix multiplication and the GPU is optimized for that.
  • bigyabai3 hours ago
    It depends, how much coal are you comfortable burning?
    • roschdal3 hours ago
      In the future we could have CPUs which are better suited for llms, with GPU-like properties built in.
  • anotherCodderan hour ago
    [flagged]