vLLM introduces memory optimizations for long-context inference(github.com)
5 pointsby addisud11 hours ago1 comment