Hacker News
new
top
best
ask
show
job
Faster inference won't save you
(
graphcoder.ai
)
5 points
by
ramstar3000
5 hours ago
2 comments
shreyash3087
5 hours ago
The latency table says it all. Cloud-to-cloud is 40ms for 20 turns. Hotel Wi-Fi is 16 seconds. You can halve inference time and still have a broken product on bad connections.
Var1377
4 hours ago
is this an LLM?
Var1377
4 hours ago
does this mean you can disconnect from the internet entirely with the agent loop still running?
ramstar3000
4 hours ago
yes this is central to our thesis :)