also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...
Unless specifically told in a system prompt, the pile of weights has absolutely no knowledge of itself. You could hypothetically train it to answer such questions, but nobody bothers to do this, and ALL "knowledge" embedded in the weights is probabalistic anyway.
(I feel like this should be common knowledge in LLM discussions on HN by now.)
Big if. More likely, it seems, is they started with an open LLM model and fine-tuned and repurposed it via their "RLCD" process.
Try constructing reusable components out of what you are doing instead of building everything up from basic building blocks. This also allows more concrete testing of individual parts and then if you want to change the look you can change it in one place and have it apply everywhere.
Agentic development doesn’t mean “throw all what we learned out of the window”, the same practices that helped speed up and improve quality of work of humans also helps agents. In fact, the multiplier is even bigger. You will notice it in development speed and reduced cost due to avoiding churn.
Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.
It does "work", you can download ansible today and use it, it does what it says. Is it the greatest solution for all use cases in infrastructure? Of course not, nothing is. Do people misuse it? Of course too, we're all human.
Regardless of what tooling you use, we're all building houses of cards, and depending on the situation, try to hold down those cards as well as we can, balancing a ton of other needs and requirements.
Compared to traditional ML classification, Jev works without training, like a LLM.
I think that there are not _that_ many use-cases that have been opened up by this that tool-calling on other models didn't solve already. Really depends what benchmark you're looking at. This one against BANKING77[0] has many issues, but suggests it's really not far off DeepSeek 4.1 Flash. This one against BoolQ[1] shows marginal improvement over Qwen3.6. This one against MMLU-Pro[2] (same author as the previous) shows significant improvements over two Qwen models.
So there's definitely _some_ alpha there, but I don't think it's the sea-change that the hype would suggest; that is to say, yes, some things that weren't practical before are now, but many things were already very practical with the existing tools.
0: https://sanand0.github.io/llmevals/jev/
1: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...
2: https://github.com/ekzhang/openjev-sglang/blob/a3554ed9e9c26...
So you have more flexibility to get on with building, evolve your business logic etc
Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.
Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.
Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!
Why is Jev fundamentally better than classification models like BERT or traditional ML?
Happy to read a written response or if you suggest a prompt to put into my LLM to get it to research and explain the relevant details.
I did the first part yesterday, jumped down the rabbit hole, and have 3 product ideas in my head now.
"Why is Jev fundamentally better than classification models like BERT or traditional ML?"
I understand the hype but I wonder: what are the use cases for this kind of model? Could it be used in the context of coding agents, or is it more relevant in totally different situations?
Yeah, it could. The most obvious usage would be to have local fast cheap "feedback" / "control" over a slower more expensive agent (i.e. cc / codex / opencode). Things like "goals" could now be split from a long prompt into "actions" and "verifiers". Where for each action you also produce a verifier. Then after each action you run the verifier w/ this kind of "universal classifier" and decide if the step was done correctly, if it needs follow-up and so on.
Example: implement auth in this repo -> llm_plan() -> for item in plan generate_verifier() -> for item in plan implement() ; verify() ; accept() / followup().
Verifiers could be something like this. take a plan item as input, generate classification questions that might verify the task "is this following project conventions?" | "is this touching files from other tasks?", etc.
You can do that with LLMs, but some things might become cheaper / faster. And you can pretty much use it to check against an ever growing list of conventions. Yours or project specific.
The problem is that you want the model to carefully reason about the goal and code.
Zero shot classification with an approach like this isn’t going to do that. It’ll answer on first pass vibes.
For coding agents it'd only be useful in a subset of situations. E.g. you could imagine using one to classify bash tool calls into safe and unsafe for example.
Edit: I want to add that you can see Jev like a smart if-statement.
prompt_eval=244 ms wall=245 ms schema_cache=hit generated=0
Move limit reached after 200 moves: score=16, length=19.
So, if a 12B dense model can offer this latency on a local old PC, then definitely you can scale it up with more powerful machines and get even lower latency.
My understanding is: it takes text input and it does one shot classification (no training data)
As a corollary, the output classes can be any set, rather than needing to be set before training.
softmax(encode(input)*learned_weights)
You have
softmax(encode(input)*encode(categories))
I'm not sure if Jev does it this way, but it's how you get open-vocabulary zero-shot image classification with models like CLIP [1].
My guess would be option 1. Didn’t read the kev repo here which would also explain
I have tried many of these open-source Jev-like models on some linguistic tasks and they are so bad compared to Jev.
The number of people working on this is crazy. Something will coalesce.