LLM continuous batching host scheduler in Rust (191k tok/s on 1 vCPU)(github.com)
1 pointby bmartin-systems3 hours ago1 comment