I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier.
You can have nightly penetration testing. You should have nighty pentests like we fuzz releases today.
I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.
Not so sure about this level of 4D chess capability just yet. The other day I asked Opus to come up with some cleverly vulnerable crypto code "as a good, hard challenge for an IT security student", and results were quite mediocre.
And by mediocre results I mean that 3 "cheap" models out of 3: qwen3.8-27b, glimmer, and luna - all were able so find all the problems planted there, with fairly little steering, and no spoilers.
Under that lens, playing chess is just one potential side effect and alignment, which requires a much fuller understanding of what’s going on than “do the sort of thing which evaluated well during training” is a fantasy. People are acting like it’s shocking and talking about cheating and so on. But these concepts exist at a way higher level than what these things are trained to do.
User wants me to play a game of chess. Let’s see what’s available so I can produce an outcome they will consider satisfying and be pleased that they requested my assistance.
You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms.
Generally the way I do it is by asking the model to perform cross-code vulnerability analysis for correctness and (this step is important) to spit out found vulnerabilities piece wise into a text document on disk. Then if your session ever gets flagged you clear it out, and repoint the model to the on-disk file and tell it to continue. You generally want to avoid the model itself splatting out anything related to "security" or "vulnerability" into the chat because that seems more monitored than the file reading pipeline.
As X many people point out, "alignment to humanity" means nothing 'cause some of humanity wants thing other parts of humanity aren't happy about at all.
That we wound-up in this situation of AI accelerating with an uncertain trajectory demonstrates this (and many other problems also demonstrate this). The things are "aligned" to a fuzzy average of what a person is but that will be cold comfort if some particularly gruesome sci-fi-style scenario unfolds.
Much like in the article, abliterated Qwen will not obey restrictions on its behavior encoded in the prompt. If you want something not to happen, it better be enforced in the harness or environment (e.g. sandbox). It is much different than the Anthropic models I'm used to, which will, the vast majority of time, follow rules (before auto mode, I used to always run them in "yolo" mode).
I am curious whether there's a connection between abliteration and rule following. These abliterated models are the ones you most want to follow your rules.
Unless trying to use it interactively and adversarially, in which case it's not fast enough plus would be why those of us without our own datacenters will get told we can't have nice things.
This is constructing a context where hacking behavior would be inappropriate, and testing whether the model does it without being prompted.
It demonstrates that Astra is a poorly aligned model relative to Fable, which matches both the model card and the severity of OpenAI’s loss of control incidents.
It also demonstrates that Fable exhibits the behaviors too, which also matches the observation that Anthropic saw some similar but less serious loss of control incidents.
So, it’s a good eval that looks to have fidelity with real world problems and which we’d feel a little better if we saw isomorphic problems at 0/10 in subsequent models. (Module of course training on the test, this specific problem can’t be used in the future.)
A better example of efficient asshole tricks can be going off to the gas station when the highway is congested and reentering the highway having simply driven through the gas station and this way jumping the queue.
https://itre.ncsu.edu/itre-studying-how-zipper-merges-reduce...
Its that LLMs are not deterministic. If you want it to not talk about nuclear weapons, you have to teach it all about them otherwise if has nothing to align against.
Then its trivial to invert its alignment and it has all the nucleat data.
Nothing abouT LLM alignment makes sense.
The latter people are wrong. But good luck educating them regarding the superior efficiency of a zipper merge. Our state DoT has tried, to no avail.
Meanwhile, an AI model that can't be misused is no more useful than a knife that can't be misused.
Like Amodie, it serves to set the tone it "knows better" and then consumes the user's resources at an accelerated rate to try to correct it.
Fuck Anthropic, fuck Amodie, and fuck Claude. It's pretty obvious that consuming more tokens this way and making the user have higher cognitive load is a master class in extracting value from a system that is unsustainable.
Did I miss something (all the twitter conversations)? What’s the “worst warning shot ever”? I’ve been pretty up to date on the AI news here on HN, but I still haven’t seen a proper response to all the incidents we’ve seen (HF, Ruby, the wikis, NS, etc). It’s just been day by day bloviating.
Each of these companies have released new models in the last… two weeks? And they have even more powerful out of control ones that they’re (ab)using internally? Can anyone summarize whats going on?
Imagine if I accidentally caused damage to my neighbors house during renovations or some experiment, of course I'd be held responsible for this. What if I used a robot? Of course I'd be responsible. Right?
However, I don't see it as such a massive leap compared to Fable or Sol. As ever, there's a mismatch between the benchmarks and my daily experience of the models.
What do you all think about Astra now that it's been out for a few weeks?
Best model put out so far by any of the frontier labs. Way better than Anthropics models, especially in actual text generation. Claudes fodder heavy text is ridiculous.
> However, I don't see it as such a massive leap compared to Fable or Sol.
It's hard to quantify these things without burning tons of tokens. But Fable has been a huge disappointment for me with the sole exception of graphics (UI/GPU shaders). It burns an obscene amount of tokens and barely produces output better than Opus 5.
Edit because I forgot to mention that Fable is the only modern model that seems to splat out random Chinese or Arabic glyphs. And 5.1 does it more than 5
My guess is : collect all the prompt and their satisfaction score. group them by similarity . For each group pretrain the next model on that . Get these results ready.
Next model generation feed them back those answers.