New KV cache compaction technique cuts LLM memory 50x without accuracy loss(venturebeat.com)
8 pointsby mellosouls8 hours ago1 comment