Instant AI Response(chatjimmy.ai)

29 pointsby hochmartinez11 hours ago8 comments

pella6 hours ago
- https://news.ycombinator.com/item?id=47086181
- https://taalas.com/the-path-to-ubiquitous-ai/
- https://www.nextplatform.com/2026/02/19/taalas-etches-ai-mod...
6 hours ago
undefined
personalcompute6 hours ago
This is a demo of Taalas inference ASIC hardware. Prior discussion @ https://news.ycombinator.com/item?id=47086181
alansaber5 hours ago
I love seeing optimised SLM inference. Is there a current use-case for this? Edge CNNs make sense to me but not edge SLMs (yet).
nacs6 hours ago
What model and hardware powers this?
Is this a Google T5 based model?
- pella6 hours ago
  3bit hard-wired Llama 3.1 8B ( https://taalas.com/the-path-to-ubiquitous-ai/ )
  - cyansmoker34 minutes ago
    3bit is a bit ridiculous. From that page I am unclear if the current model is 3 or 4bit. If it’s 4bit… well, NVIDIA showed that a well organized model can perform almost as well as 8bit.
Kuyawa6 hours ago
If this is possible, why not all online AI engines work like this?
- yomismoaqui5 hours ago
  This is an specific model (Llama 3.1 8B) baked in hardware form. You can only use this model but get "low" power consumption and crazy speed.
  If you want to run a different model you need new hardware for that new model.
  - sixtyj5 hours ago
    It is really a crazy speed. 15k tokens/second.
    sixtyj15 minutes ago
    I have tried it again. This is the future of chat UI, imho.
    Generated in 0,074s • 15 754 tok/s
OutOfHere3 hours ago
Impressive, but this particular underlying LLM is objectively weak. I'd like to see it done with a larger and newer better model.
notronic5 hours ago
imagine a model like opus 4.6 at that speed, that would be insane