Here's a quick demo of Glaux on YT: https://youtu.be/Uf8GytVp5sw?si=yebS0mmSUjt0EuO7
My project is called "Glaux" (after the owl of Minerva). It's a Next JS application that (tries it's best) to download, validate, and inference HuggingFace ONNX Community models utilizing Browser Storage/WebGPU via ONNX web runtime and Transformers.js
The project started when me and my friends wanted to build our own AI workstation to experiment in the field of ML systems engineering. Unfortunately, we didn't realize how expensive and complicated it is to set that kind of rig up for a frontier OSS model. However, that made me wonder what I could run on my personal laptop, which led me to discover a whole plethora of browser ML system libraries/technology such as ONNX, Transformers.js, WebGPU, etc.
Since then, I've spent about a month hacking (prompting) away at Glaux and I'm pretty proud of the build so far. The design philosophy is relatively simple: No accounts, analytics, external inference servers, and best effort transparency. But the implementation was a lot tricker than I thought.
You can check out the GitHub here: https://github.com/rangan39/glaux
The GitHub contains more context on the technical specs, including architecture diagrams, technical limitations, and a more in depth explanation of some of the implementation details. I'd really appreciate and am very open to hearing your thoughts and technical feedback!
- GR
Looks really impressive. What would you like to see people build with this?
Personally, I see it as an educational tool for ML systems, I'd love for people to try and build features that get it closer in parity to Big AI applications.
Gaurav it’s great to see you posting here - sweet tool.
Modeling running from browser is awesome.