Hi HN! I'm the author of mere.run a local first inference runtime built around an installable CLI. I believe that whenever possible we should use the stuff we already own (like our Mac laptops, decent machines gathering dust, our gaming PC) and the limited electrical power we have easy access to, like the socket in the wall next to most of us. We shouldn't have to send our data to the cloud hoping some T&C will prevent it from being used in a way that we'd regret. Most of the local AI solutions are technical, involved, and land a curious body in some package hell. People are optimizing for one system and not another, the fun stuff is on PC if you own a Mac, and on Mac if you own a PC.

So mere.run was my choice to begin to patch many of those things that I saw as problematic. Text, chat, code, image gen, speech tts+asr, vision (caption/ground/segment/track/pose/depth/face/OCR/etc), music, sfx, video, 2d->3d, persistent worlds, lora training plus a few more things I am probably forgetting all in one place to work the way you work, with a scriptable CLI, an openAI compatible serve, and optionally a native app on Mac.

It's native swift on MLX, no python, PyTorch, diffusers for inference. For text lanes with GGUF it uses llama.cpp and for a/v muxing its FFmpeg. Most upstream releases that don't have an MLX variant are converted offline and hosted on huggingface. All release packages are built for arm64 (Mac & Cuda) plus x86. They're signed and ship SHA256SUMS. No windows at the moment.

> mere.run model capabilities --recommended # That inspects your machine before recommending to prevent pulling models that don't fit your spec

There's a workflow layer with typed, validated graphs so you can create immutable job bundles and run them locally, over an ssh executor or using a fleet of machines and the relay service. (It's hosted at relay.mere.run and is currently invite only while I test, but its also totally open MIT so you can set it up yourself)

The whole runtime is MIT along with the companion packages, models carry their own licenses and the CLI makes it clear when something has specific terms. Once you've pulled the models, everything works fully offline.

I've been working on a (hopefully) comprehensive docs -> https://docs.mere.run

No account, no API key, no analytics or phone home. Any network calls in the source are all at your request only like huggingface.co (models), GitHub.com (Pi install).

I'm just getting started, but it's finally at the point I'd love feedback, contributions, and just folks to generally find it useful. I hope it helps you make things and explore what's possible with the stuff you've already got in your home.

Would love any thoughts, questions, ideas and maybe a star if you do that kinda thing.

-Kyle