Proxying inference requests in 6ms with Pingora, Envoy, and Spanner(modal.com)
4 pointsby birdculture7 hours ago1 comment
  • prios86 hours ago
    Really interesting approach. How do retries work if a stream drops halfway through?