Hacker News
new
top
best
ask
show
job
Using KV cache as embeddings
(
breadbowl.ai
)
2 points
by
mxudev
2 hours ago
1 comment
mxudev
2 hours ago
What if instead of one giant vector as embedding, we use multiple (K, V) pairs. In this work we demonstrated this is feasible, and got reranker behavior at retrieval cost (without extra backbone pass)