I can't recommend LocalAI enough. I am thoroughly enjoying it in all aspects. Especially the memory management and ability to load and boot models to and from memory as needed, right from the clients that use them.