Hacker News
new
top
best
ask
show
job
Show HN: Relay – a self-hosted LLM gateway with smart routing and request pacing
(
github.com
)
2 points
by
pavelmelnichuk
5 hours ago
2 comments
codaws
13 minutes ago
how do solve KV caching problems routing causes? if you route to a different provider you'll have to re-prefill, no?
pavelmelnichuk
4 hours ago
Hey everyone, I'm here to answer questions all day if anyone needs anything or wants to know more about how I trained the classifier, also you can find docs at the website: anchorshell.com