There are a couple of options. One good way to find inference providers for open models is through hugging face (https://huggingface.co). you can select a model and see which inference providers serve it. you can even access it through hugging face.

If you just wanted to test a model or have super light work you can get some free access to alot of open source models through nvidia (https://build.nvidia.com).

There is also Runpod and a few more.

Is there something in particular you need? if its just cost/performance/speed, check hugging face for there inference providers, it compares them against these three.

Although it's closed, openai's luna model is a great balance between cost/performance/speed.

lastly if you wanted to help shape a new inference provider, I am working on a more pro/consumer focused platform. It has a nice iOS app, platform etc. Models are limited currently and it's in beta. if you want to take a look it's called Ando and can be found here - https://andoai.xyz. (Its very early in its build, it will break, any and all feedback is appreciated)

For LLM inference in production we use vLLM (with our own model).

[flagged]