Overmind: Cut PyTorch model load time 15s to 0.2s with a zero-copy shared cache(github.com)
2 pointsby westurner2 hours ago1 comment
  • westurneran hour ago
    meshy-dev/overmind: Cut PyTorch model loading time from 15s to 0.2s with zero-copy shared memory caching.

    > Fast Model loading: First load caches to shared memory; subsequent loads are ~5x faster

    > Process-agnostic - Cache persists across process restarts via a background server

    > Non-intrusive - Just add one line of code; no changes to model loading logic