Eval context: 30 bug fixes and features evaluated by tests the agent didn't see. Ran on Opus 5.5 at medium effort in Claude Code.
20 of the tasks were last run on Nov 1, twice each. Nov 1 solved 36/40, and today solved 19/20. Thinking tokens per task were 0.95x Nov 1’s and output tokens 0.97x. The one significant change was speed: median time to first token decreased from 3.0s to 1.7s.
Caveats: one run per task today, medium effort only.
Paragraph 1 says there were 30 bug fixes.
Paragraph 2 first says they tested it on 20 of the “tasks”. But why is the denominator 40? Where did that come from? It should be 30.
It goes on to mention another test, this time with 20 tasks. Again, it’s unclear where this is even coming from. Based on the first paragraph it should be 30 not 20.
Why paragraph 1 even mentions the full set of 30 tasks if 10 of them are irrelevant to the comparison, I don't know.
I'm not sure if it's LLM text, or somebody who has been reading way too much LLM text and is now corrupted into mimicking it. My guess would be a mix of human and LLM. "...evaluated by tests the agent didn't see" sounds very Claude. The first line (2 sentences) sounds human to me. Everything after sounds AI, though the last line is ambiguous.
It's kind of sad how our meat brains are getting non-consensually trained on this categorization task....
https://news.ycombinator.com/item?id=50422951
So only if you were "selected" for their experiments, your account was nerfed.
I’m working on a task. And the task is never to evaluate the agent performance.
If the nerfing is lower effort (seems to be, presumably why they hide the CoT) then they may be wondering if they can get away permanently with lowering effort on the new release provided it gives a wee improvement.
Sometimes you just have insufficient evidence to be sure of something either way, and nobody gets to win the argument or shut somebody else down.
In this case, some have reported analytical work[0] that goes beyond "gut feel" anecodotal evidence, but even that's not going to conclusive and may not be trustworthy at all.
[0] https://www.reddit.com/r/ClaudeCode/comments/1xr9x5y/the_ner...
They don't detect a quantized model until the quantization gets down to 4 bits or less.
Humans absolutely do detect quantization well before 4 bits...
Try it yourself if you want. Try a small local model, both quantized and not, and I would be very surprised if it isn't immediately obvious which is which.
But no, evals and benchmarks do not detect quantization, until around 4 bit quantization.
https://developers.redhat.com/articles/2024/10/17/we-ran-ove...
https://pub.towardsai.net/your-model-passed-safety-evals-the...
so not everyone will have the same experience. If I was saving on costs that would be the easiest way to do it without deploying, without any nefarious intent. The persons in charge of that likely don't even know about how individual users are impacted
Or switch to a faster, cheaper cache mechanism or something that's almost as good, but much cheaper to run, in the name of improving efficiency and keeping up with demand. This switch to cheaper mechanisms is what it feels like to me.
For example a provider quietly underclocks your cpu or puts it is an energy saving plan hoping you won't notice.
https://www.reddit.com/r/ClaudeCode/s/64DXRERLyU
It would be nice to know how they select. I max out my $200 subscription every week and I was selected. On Saturday, the Opus on my account that happened to be supervising a running live process, became unusable.
I really really really do not trust companies on the verge of IPOing from goosing their numbers. Even if the lawyers are up their ass, the social pressures are more than enough to compel the behavior. For every good person willing to do the right thing even at the cost of a delayed career progression, there are 20 people willing to do anything not even for a dollar but for an "atta boy!" from the boss.
Max, streamed (128K cap): 128,000 tokens, $2.56, empty file, in September and today.
Max, Batch API (300K): 255,957 tokens for $2.56 in September, 243,707 for $2.44 today. The two drawings: https://playcode.io/marketing/assets/static/claude-opus-5-5-... and https://playcode.io/marketing/assets/static/claude-opus-5-5-...
Xhigh: 86,096 tokens for $1.72 in September, 107,333 for $2.15 today.
The Batch API bills half price. Max ran out of the 128K ceiling again, to the token. Given 300K, it drew the same kind of laptop with 5% fewer tokens. Xhigh used 25% more and drew the same kind of laptop. That's inside the noise: 17 identical calls at high ranged from 17,757 to 59,705 tokens.
The model is still the best of everything we've tried. Most likely the next best would be Fable 5.2. But we don't like Fable because essentially the result is the same, just the price is higher. Its medium and high are close to what Opus 5.5 Max had, but the price is higher.
(I run Playcode, an AI app builder. We benchmark every model before we add it.)
I doubt this is the case. I suspect all variation in quality and output can be explained in terms of prompt differences, and changing how much thinking happens at each level, as we see in the post above.
And what about A/B testing? Anthropic doesn't even try to hide that they do this. Limited compute and A/B testing alone, would guarantee a nerfed experience some of the time. That is, you don't even need a nefarious reason.
I consider it gas lighting to deny model nerfing.
They do drop requests though? At peak times I sometimes have 1-2 requests fail with Opus; no biggie, the harness just retries it. But that, in aggregate, sheds load.
Thats not what Openai and Anthropic do though
If you want to go accuse someone of denying model nerfing, go find someone doing it and argue with them.
Edit: maybe not impossible. Side channels perhaps.
Oh, this Opus X.Y is incredible. Hmm... Now it's dumb. Rinse, repeat.
Doesn't really matter if the cause is quantization, system prompt changes, changes to thinking etc.
It's hard to see how continuously rug pulling users at every release is a good idea. Only reason I can fathom is that it's needed to keep costs inline.
Either that or it's some weird tweaking to the prompts to fix whatever. But given the prevalent sentiment that it's worse that sounds like a long shot
but realistically I think you're right, it's prompt quality and the inherent non-determinism elements of LLMs showing up; when both of those things meet you get brain dead outputs.
Everyone likes to shout "slot machines! non-determinism!" until it actually explains something, heh.
This is getting ridiculous. The number of times I have switched from one model back to the other is crazy. I do not have any proof, so its quite possible I'm just delusional. AI psychosis? Either way, it FEELS like these models work really well for a while and then fall off. Switching to the latest model on the competitor always seems better, until it too seems to taper off. By then, a new model is usually available at the competitor, so I switch.
Its possible there's no degradation at all and just a PERCEPTION of it. But to me the end user, it really doesn't matter.
People seem to underestimate how high normal variance is with these tools.
If this turns out to be accurate, I think that would be a pretty unethical thing for Anthropic. Even if it is just a "we had to allocate compute to this new model release".
It should be fraud to swap out the model. We need a term for this, something like a bait-and-switch.
The media coined the term "dual quality" for a European food scandal a few years back, in which conglomerates (including Coca-Cola) adulterated products with cheap, substitute ingredients, if they were destined for distribution in specific markets (mainly, lower-income).
The packaging exteriors were the same; and there were (I think)? broadly no price differences. It was essentially what the software industry calls "A/B testing", except with sneaking things into what people eat.
I don't have any good reading links on hand; here's one source,
https://ec.europa.eu/commission/presscorner/detail/en/qanda_... ("Dual Food Quality: Questions and Answers")
I'm a bit disappointed with how this nonsense is spread across HN today.
Are you saying that the many, many people posting and experiencing this are wrong and you are right?
And not just in this instance, nerfing has been complained about in a cyclical fashion for many months.
But, they're all wrong?
You're so confident in your experience that you cannot think that other's experiences are also true?
No, it gets complained about constantly even if there is no model coming out. A completely unrelated thread on Claude will feature comments like "can I just say that [model] has suddenly become useless this week?" Merely asking that question is bound to get people agreeing with you because we are evaluating all of this on vibes and placebos.
I too feel that way sometimes, I don't think it's ridiculous that Anthropic might silently downgrade you as demand warrants. But asking others if they "feel" something I feel is less than useless.
"Could all these people seriously be wrong?" Yes, yes they could.
if they just straight up dont sell you something you bought then its fraud.
I think there is a ripe opportunity for expanding the definition of fraud to include changing the "ingredients" of a named product on the market. It is not limited here, we see it with food, and even in the medical field
(I will die on the hill that medical fraud was committed when mRNA therapy was used as a treatment for covid and it was 'sold' to the public as a "vaccine" despite not being the same thing as any vaccine before it. The definition of vaccine was expanded to include it without sifficient public disclosure to warrant informed consent imo. This is not a statement about conspiracy or the efficacy of vaccines or a moral judgement on whether it was for the greater good or not.. it was simply an act of fraud that robbed the public of proper informed consent by making people believe it was X when it was really Y.)
So yes, I think there is a large need for the public to at the very least put a strong handle on the ability to change what a product name means. It is a trivial problem to solve considering it is only a problem due fraud or fraud adjacent activities.
heres the solution: You have your product name, and then any changes to it REQUIRE a new version name. Like how you have Coke and Diet Coke, only formalized such that Coke cannot legally replace the ingredients of regular Coke with the ingredients of Diet coke and still call it Coke. With Software they could at least use version numbers and release modifiers. most of the time its built in already, but it needs to include the full stack. if they change anything that has an interaction with the user it should change the label seen by the user for the product.
thank you for coming to my ted talk
A Opus 5.5 is not a Opus 5.5. Its Opus 5.5 of some checkpoint.
mRNA btw. was well known how it worked, it wasn't hidden and some people waited for the non mRNA version because of it. I never seen or read the argument you make and I do not get it.
A vaccine is something which protects you from an infection. What definition do you think changed in humans? That we make the protein? How is that so different from viral vector vaccines?
The consent is a certain level of trust from the society to a group of researchers limited by being humans and having to act in a certain time frame.
Nontheless, I do agree I would like to have a fingerprint and proper changelogs etc. for products I buy including digital and analog. But this has a downside as usual: The smaller you are, the more it costs you to (probably) register, maintain and change.
You know the good thing about Claude etc. right now? It doesn't matter. My invest is not 4 digits and there are that many changes, i'm quite happy to have been upgraded for 'free' from Opus 4.8 to Opus 5 (what do you call this upgrade, which was significant better?)
Fuck Anthropic.
Opus 5.5 was so good that if it wasn't nerfed - at least just looking at the benchmarks - the difference in cost here does not seem to justify using this for anything besides design and review.
That's like maybe 1% of total token usage for SWE BUT a better design can have enormous benefits, so it's almost always worth using the best model for that.
System prompt would be the easiest one.
But there were flash or inbetween models, different checkpoints etc. THey might also try different hardware which might not be binary compatible.
Anthropic does this time and time again, unserious company
https://x.com/follistatindev/status/2119129611510383014
With early handouts of Qwen4 starting to be reported 5 days ago ... so if they're good, they would have had some hours to evaluate Qwen4 against Claude 5.5 and, then or right after was when Anthropic decided to put out 5.2 ...
So yeah ... benchmaxxing and all that. BUT nevertheless, the published results seem to say that Qwen4-Max appears to match Opus 5.5 on some benchmarks, and Qwen4-27B (probably by going even harder than Qwen3.8-27B on thinking/inference time compute) can beat Sonnet 5.5 on some things.
I guess they're not taking any chances. Keep in mind that Anthropic has declared they have non-public models (it's in their IPO filing). These are referred to by the shorthand names Model 2, Model 3, and Model 4. Essentially 4 is better than 3 is better than 2.
Fable 5.2 is "2.5" so to speak. So Anthropic was holding back, but not anymore. And now they can dial the performance up once more with what they have ready, and probably once more after that.
It would be funny though if you are right and that model is Fable 5.5 in a couple weeks.
Most people don't really have intuitions on what it means to push the frontier now (because AI has surpassed everything they are good at, for example, coding). We are truly inside the singularity (using Kurzweil's definition, slightly paraphrased - the time when technology progresses so fast no one can keep track with it).
What we're seeing instead is maybe akin to the green revolution. Before nitrogen fixation nitrogen was a resource constraining us humans. Lack of grit is what kept us from solving mathematical problems. However this lets us run into other resource constraints. A lot of technological progress does not solely hinge on thinking/processing. The speedup of creating new medicines with LLMs will probably be very small, as you still need to do all the testing. In software development you still need to first find out what to build in the first place and even mathematics where LLMs seem to shin brightest, it's becoming clearer that mathematics is not just solving mathematical problems.
Apart from that, a good coder is better than LLMs, shown by the amount of hand-holding needed. The only issue is that the amount of research an LLM can do and knowledge it is trained with far surpasses the amount of work humans would need to spend by hand, but we can also us LLMs for that.
IMO, it is very interesting and also very alarming. At some point the barrier to the exponential progress is the amount of computation available. I would consider hitting this limit and then likely using the same said swarm to help resolve it via various means as very much fitting under the definition of Ray Kurzweil's singularity.
Also, I would disagree, Fable is a sufficient coder. The average human being better, even if that is true now (and personally I wouldn't agree), the average human is limited in the number of 'human instances' he or she can interface with at one time.
if 1000 fable instances is better than any human at a task, how about 1 million instances running? Much easier than finding and hiring and coordinating 1000 extremely productive and intelligent human beings. Much cheaper too, it appears.
Like models getting significant better every 7 month and model usage on openrouter.
https://metr.org/ << someone said though that they are biased.
Math is for sure about solving mathematical problems, mathematicians complaing about humans capability of understanding it. If you need math an LLM understands, it might not be relevant if you understand it if an LLM is getting a lot better in adjusting itself if it goes the wrong way.
And this situation is only a few weeks old. We don't know how this field continues to progress. Books like the culture series and other scifi do describe a future were humans spend their time only to understand what the machines are doing or did.
The speedup of medicines is for sure faster with. There are still old labs doing everything by hand and the neolabs doing everything a lot more automated. An AI can do experiments 24/7 and can scale with the hardwarelabs, but it also can hold the whole research field in context and it might be a lot faster of choosing the better experiments or the right ones to begin with.
In software developmet (my field) i can do today alone what you needed a Team for. Yeah it doesn't help the field of software engineering if you need 5 times less people and you don't have any motivation AT ALL to teach someone avg. We are already planning to remove one member of our team due to AI. And we will not extend the contract of 2 consultants.
It just doesn't make sense anymore. If i explain in a ticket a task well enough for someone who has not been in our team for years, the AI is already better in it. We don't struggle with AI output, we struggle more with not having a Monorepo.
Give an LLM access to a GPU system or a GPU cluster. It immediadly feels complelty different then running it on some laptop. It can now do simulations, a lot more and faster experiments etc. I had this epiphany just a few month ago. I always used my laptop for projects (it needs very little ram, i can quickly check what it is doing and i actually do use it primarily when im on holidays) until I thought "let me use claude to do some gaussian splatting experiment its always so cumbersome to use these tools" and it changed my perception immediadly.
I don't think that case can be made. I'm a "good coder" and the best LLMs are now better at it than I am, in essentially every respect that matters. I can't think of a test that I could reliably pass that Fable or Astra would reliably fail.
Model AECI 95% CI (global)
Claude Opus 5.5 174.78 170.33-180.29
Claude Mythos 5.2 177.12 172.42-182.94
Internal model 178.78 173.75-185.18
In H1 2026, cost of revenue is $2.3B. That is primarily inference compute, but also includes cost of training models (but not research).
In the same period, R&D was $9.4B. That is primarily spent on compute. That is what they are "holding back" already, in full public view.
This is not a company that is optimizing for OpEx or end user revenue. The only thing relevant to their plans is achieving AGI/ASI first, and controlling enough compute to leverage it to earn the majority of the current economy.
Now imagine feeling like this, except not just about "AI events" but about everything around you.
Genuinely curious what that costs in terms of energy input and amortized computer cost vs. the previous version + a human's salary for 11 days.
We've sort of entered this twilight zone where most humans don't have the time, expertise, or moral compass to be able to use these models.
We're all going to be harness engineers in the future.
Within three years most software developer jobs will consist of taking instruction from the AI systems. The humans won't see it that way, but that's what it will actually be. Realistically the thing that will happen is the humans will be put into harnesses (cubicles) and prompted by the AI (I need you to plug that server in; I need you to give me access to such and such; now do this, then do that). Most tech jobs will consist of AI telling the human what the AI needs the human to do next, de facto the human will be in a harness and will be getting told what to do.
I first heard about this in Marshall Brain's short story _Manna_ [1]. In the story, the robot takeover started with AI management software running fast food restaurants, where workers would execute tasks the AI would narrate to them over headsets.
I bet we're pretty close to being able to implement this.
1. https://web.archive.org/web/20260725174355/https://marshallb...
An algorithm isn’t. I still dunno how they’re gonna square that circle.
The state of the art doesn't matter. Only the first couple of time derivatives matter.
Someone has to type in "create the next billboard top 1 single" ?
So whether that's a CEO at the top instructing a human manager that then instructs AI systems, which then put into motion plans that include the harnessed humans getting prompted, or however it's decided to best configure the top operational tiers - that's all fairly irrelevant ultimately. All human labor, and all AI labor, will be in harnesses. Realistically all human labor has always been in a harness.
The people in charge will be the people that control the capital. They will instruct the AI to their bidding, they will control the data centers the AI lives in, they will own the robots doing the labor in factories, they will own the factories, they will control the grid, they will control the electricity generation, et al.
It's true now, it'll be true tomorrow. Nothing fundamental will shift at that tier: don't be labor, own.
If you're a software developer, you're increasingly depreciating labor. Every unit of improvement the AI systems see, the average human software developer's labor value diminishes. It's diminishing extraordinarily fast, kind of like a confrontation between increasingly productive machines and physical human labor during the industrial revolution. You might as well picture a human with a hand plow taking on a modern industrial tractor.
Sounds a bit like self aggrandizing elitism to me. I can imagine some self aggrandizing nerds saying the same thing 50 years ago:
> We've sort of entered this twilight zone where most humans don't have the time, expertise, or moral compass to be able to use these IBM PCs.
Now everyone use computer.
No, that will be the model's job. We will largely be unemployed. Of course, we can surely depend on the largesse of Altman and Amodei to keep us fed and sheltered.
Lolwut. If this shit works the way it's boosters claim, humans will have absolutely nothing to contribute.
Sit down, shut up, and get out of the way, because your involvement is slowing shit down.
Robotics are lagging, so there's probably a few years where we're going to need humans to crawl into that awkward space under the counter and twist some knobs, or to clean toilets.
I would assume if they could get twice as much progress by using 10000 simultaneous running fable instances instead, their brief would state that.
IMO, it is very interesting and also very alarming. At some point the barrier to the exponential progress is the amount of computation available. The average human is limited in the number of 'human instances' he or she can interface with at one time.
if 1000 fable instances is better than any human at a task, how about 1 million instances running? Much easier than finding and hiring and coordinating 1000 extremely productive and intelligent human beings. Much cheaper too, it appears.
We are very close to some very scary things happening. I just recently came to terms with the fact that an AI could code and manage coders faster and better than me, I don't think anyones is ready for what is about to happen.
Where it gets scary is the fact that efficiency is going up 10x per year, so the cost of running 1000 agents will cost the same as running 1 agent in 3 years if the trend holds.
AI discovers 500 things per day.
People mine them for interesting results.
Devote precious time (in the face of tons of discoveries) to validate the most compelling ones.
If agents actually can continue to produce tons of results with real world implications, and they arent generally found to be shit, I could see that becoming the job of experts. AI discovery validators.
Can they produce more code more quickly? For sure.
Can they go deeper on a topic instantly? Definitely.
Would I say their output is better? Almost certainly not.
At this point it seems impossible for human to write better/more efficient code than a swarm of agents -- to be clear, multiple coordinated LLMs, not one-shot builds with a single agent.
A frontier AI is at least as capable as ten median human software developers. It's likely well beyond that now. Over the course of a year, would you have to spend $1.5 million to get the output of those ten median humans? No chance in hell. You'd have to spend less than 10% of that.
Then on top of that: the AI is improving rapidly; the humans go on vacation, the humans decline, the humans get old, the humans get sick, the humans take parental leave, the human struggle to adapt, the humans are resistant to change, the humans riot, the humans protest, the humans need healthcare, the humans want paid more, the humans can only work for 4-6 hours at peak while writing software (slowly).
Do you know any humans that are experts at a dozen programming languages? That human does not exist. The frontier AI are omnimaths.
Im sorry... what?
You are replying to this:
> Our first formalization of Fermat’s Last Theorem, performed by an unreleased research version of Claude, took 11 days. It relied on a purpose-built harness along with guidance from a human researcher. By contrast, a team of 1,000 Fable 5.2 agents fully formalized the theorem in just 30 hours, with minimal human intervention. Fable 5.2 started from minimal prompts, built its own ways to coordinate, and used no mathematical references, only a list of intermediate theorem statements to score progress. It needed one-third of the action tokens as our first formalization and produced a proof seven times shorter. As can be seen in the figure below, Opus 5.5 (with the same 1,000-agent setup) completed only 26% of the formalization in 18 hours, and barely made any more progress after 30.
It's all going away. Here's a prediction: HN will effectively die with the software developer.
Is this really so astonishing, assuming that the 5.2 training data certainly had the 'traces' from the Fable original run in it's training data.
Had Opus make software for some local business and seeing it blew people's mind.
20$ aint cheap for us but I'm actually considering upgrading to the 100$ tier.
Am just jelly people are using 100-200$ subscription ^_^
Sub 1% of the planet has any practical use for the top tiers of AI systems. That will likely always be true.
The gap between the haves and have nots will drastically widen further, and very quickly. The bottom ~95-97% will be left behind in a way that will be impossible to overcome. The age of AI + robotics will just make it that much harder to climb for most people, these are systems that are in aggressive, direct competition with human labor. And in short order the AI + robots will be vastly superior to human labor (it doesn't really matter if we're speaking in terms of 10 years or 30 years).
AI will consolidate the winnings. The gigantic gains by the elite rich related to AI over the past few years, is merely a hint of what's to come. Capital will win as it always does.
There will be some gains for most everybody else, those gains will be comparatively trivial though. A house bot that saves people an hour of labor here and there, AI that automates some trivial life tasks.
AI has no money, it has no wealth, it owns no capital. For now. If you want to avoid the brutal human vs AI/robot competition arena: own capital, own the AI, own the robots.
You're obviously not a frontend web developer.
These upgrades are easy peasy. "oh, my developer chatbot is smarter? great"
Like, we're dropped into a post post post singularity world where AI already did all of that and is banned? as kind of a footnote that's never further explored in movie renditions that have been released
The semi-canonical Dune Encyclopedia (disregarded by Frank's son) differs vastly from prequels and generally points to it being a self-inflicted wound on humanity that opened the doors to widespread return of feudalism and essentially stopped humanity for a 10,000 years
Uh, are you serious?
and just about "we cannot really predict beyond certain things" (Vinge's "Singularity" (1993) with its "opaque wall across future")
No they were exactly about superintelligent AI!
Vinge's 1993 article literally opens with this!
> What is the Singularity? The acceleration of technological progress has been the central feature of this century. We are on the edge of change comparable to the rise of human life on Earth. The precise cause of this change is the imminent creation by technology of entities with greater-than-human intelligence.
https://frc.ri.cmu.edu/~hpm/book98/com.ch1/vinge.singularity...
In fact, one of the options Vinge considered was a "group intellect" formed from how humans communicate more thanks to internet
Stan Ulam paraphrased John von Neumann as saying:
One conversation centered on the ever-accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.
Von Neumann even uses the term singularity, though it appears he is thinking of normal progress, not the creation of superhuman intellect.
I mean we don't even think about it but currently we are sort of discovery starved. We are fairly able to keep up with the theory. If a new technology is discovered we can implement it before something better comes out. What happens when our N discoveries per days becomes 1000N discoveries per day?
It flips. We now have more knowledge than we know how to capitalize on. We have to search it. Mine it for opportunities.
And then we automate that.
So since they are doing that task anyway (re-solving the problem increasingly more efficiently), this is a easy check point to publish to show how far they've gotten in that process.
Kurzweil probably didn't know the singularity is akshually 1000 schizo monkeys banging on a billion dollar typewriter.
It had seemed like progress was slowing last year as the juice ran out on moar parameters. We're well past that.
- Anthropic is finally well balanced across models. Meaning: Opus 5.5 is so good that I do not feel Fable FOMO as I did when it was Fable 5/5.1 vs Opus 4.8/5. And Sonnet 5.5 sometimes makes me forget about Opus.
- Fable is good, but you really have to push the ambition of your prompt in order to see proper separation from Opus 5.5. I personally have a hard time telling the difference between Fable 5.2 and Opus 5.5 for ordinary coding tasks.
- It’s way faster and more reliable: one of the worst things about fable 5.1 was how slow it was an, on top of that, how frequently it would suffer from max output token overflows. I got none of these issues with 5.2.
- Fable is exceptional at drawing. It’s ability to render an image with JavaScript actually has me questioning the value of dedicated image models. I prefer the images I’ve gotten fable/opus to make for me over those from gpt-image, it’s wild to think we’re at this point.
I love seeing this verbalized. We are pretty much at the point that increased model capability will only have utility to folks that are willing to let go of even more understanding of whatever the model is producing.
The best models can/will produce a year's worth of output that would take several days to weeks to properly understand in the span of an afternoon.
For everyone that still wants to feel like they own the end result, just use the lesser models with less ambitious tasks.
Caveat: I think this mainly applies to coding. I would still like the best models possible to be working on e.g. new drug research to cure our various ailments.
I’ve been shifting away from using more traditional GenAI models in favor of just giving LLMs access to a custom drawing language (LOGO-like) to generate imagery and its been going shockingly well - Opus 5.5 in particular.
I put together a simple LLM driven skill that creates old adventure game (EGA) environments from simple pencil sketches and it works really well. And being procedural, they're much easier to make surgical changes to without degradation.
Not just "iterate on this XGBoost model" but actually looking at a problem and designing and carrying out a short-term R&D program including probability modeling, model fitting diagnostics, how to respond to various kinds of results, how to interpret/reason about what it finds, etc. That's really where I saw all the models perform poorly, and where I saw the first big step increase from Opus 5 (pre-5.5) to Fable. But then Opus 5.5 came out and I haven't needed Fable since.
Mind you, I'm doing slightly different work (and I'm very much "in the loop" still) than when I performed that side-by-side test. But it's really amazing what Opus 5.5 can do. I now have a powerful data science AI assistant that not only can write any script I need it to (including some scripts I didn't even realize I needed), and it has an encyclopedic knowledge of everything ever, and it constantly checks me on oversights, it handily derives expressions and clever model reparameterizations that aid, it has great insight into model behavior, and it has an amazing ability come up with ad-hoc experiments and tests that would have taken me 2x as long to even think of (let alone implement by hand).
It's not at the scared-to-lose-my-job phase yet, but it's become a tremendous force multiplier for a smaller data team.
I wish I had a bigger N so I could rely on that experience a bit more, but it sure does feel like I'm not really contributing anything in terms of "Data Science expertise" in my own research right now.
i also wonder if they're not losing motivation due to the fact that the llm can race through everything so swiftly
But then I'm someone who hasn't found Opus 5.5 good for much beyond web UIs and related frontend development. I find it more forgetful than GPT-5.6 or 6.1 or Fable, so despite having a 1M context window it gets lost in a big codebase or goes of target. Opus 5.5 doesn't manage agents particularly well IMO.
I'm surrounded by people raving about Opus 5.5 and God only knows why I'm not getting their results.
The bitter lesson strikes again!
I personally have consistently been disappointed by any artwork/assets created by Fable and Opus, at least compared to Astra and Sol.
- How did they produce such a bizarre effort-intel curve for Carbon that goes against every available benchmark?
- Why did they leave Carbon-- positioned primarily as a cyber model-- off of the single cyber benchmark comparison?
- Why do they switch to comparing against Argon (rather than carbon) for robodojo?
Because on that specific benchmark (FrontierCode 2.0), Medium is Carbon's best effort level and xhigh is Fable 5.2's best effort level. You can see that on the graph your next point is about.
And it's just that one benchmark that's at medium. "All scores show each model’s highest-available effort unless otherwise noted".
> - How did they produce such a bizarre effort-intel curve for Carbon that goes against every available benchmark?
The graph is labeled as being about FrontierCode 2.0. Those appear to be Carbon's scores on the official FrontierCode 2.0 leaderboard: https://cognition.com/blog/frontier-code-2.0
> - Why did they leave Carbon-- positioned primarily as a cyber model-- off of the single cyber benchmark comparison?
> - Why do they switch to comparing against Argon (rather than carbon) for robodojo?
Because no numbers are available for Carbon for those benchmarks, and the model isn't publicly available so they couldn't run the benchmarks themselves?
And from your followup:
> where's Astra on that CVE chart
To run Astra on a vulnerability finding eval, you'd need to be part of OpenAI's verified cybersecurity program. Otherwise the results would be tainted by refusals. Do you expect that OpenAI would grant such access in this case?
I am referring to the chart that cross-compares multiple benchmarks, which Artificial Analysis clearly shows Carbon-high as superior on. Further, AA's FrontierCode chart places Carbon-high as having a higher score on Frontier 2.0 than Carbon-med (https://artificialanalysis.ai/evaluations/frontiercode-2-0?m...)
Where it gets very interesting is when you look at the Cognition "FrontierCode" publish date (2026.10.27), and the revision as of 10.28 in web archive (https://web.archive.org/web/20261028175237/https://cognition...). This revision has a VERY different Carbon effort / intelligence curve where high beats medium.
It is only with the latest revision, as of today, that all of a sudden Carbon's benchmark curve goes cone-shaped. Curious, indeed....
I'd encourage you to take a look at a third-party's take on this and see whether Anthropic's self-serving report looks "on the level" after doing so: https://artificialanalysis.ai/?models=claude-fable-5-2-high%...
Do you expect that nobody at Anthropic has such access? Trillion dollar company can’t manage this basic level of opposition research?
https://www.youtube.com/watch?v=NO2I3ueVQrk
And they have one on fable 5.2 that was just released, yay
But I suppose the reality is rather obvious when you note the funny games they're playing with the graphs, and the things they won't talk about (where's Astra on that CVE chart?)
I've been using Opus 5.5 on xhigh for some decently aggressive software development on a ~125k-line project, and it has been fine for me. I'm not really willing to dive into a "rewrite the whole project, don't make any mistakes" level of refactoring. I'm not looking to try to prove Fermat's Last Theorem myself. Is there anything that I'm truly missing?
Opus 5.5 very frequently gets into rabbit holes and imagined edge cases which I have to intervene and tell it to ignore. It ends up taking longer and costing more tokens. It also very frequently requires me to agree on how to move forward multiple times, whereas fable just drafts the PR and tells me its ready. I have steering included in the skills to avoid some of this but Ive seen these same patterns frequently in regular opus 5.5 use in other areas as well
But the improvements in how well the model does the work are noticeably better with every good model release. As of now, orchestration of agent pipelines is ALMOST so good that I can point it at a problem, and it will solve it safely and reliably without making a mess. But it requires an agent pipeline with context gathering, refactoring, attack sessions, a sprinkle of Sol 6.1 for bug hunting, and Sol 6.0 for Daybrak security audits, and a final de-bloat/clean up pass. I’m feeling that we’re getting close to doing a lot of the work fully agentically. But I can absolutely "feel" that we are not there yet when it comes to the intelligence I need. I need moar!
I am also happy with this current state, as I believe it will at some point allow me to transition to open models once they reach the level of opus 5.5. high. But its probably futile to try predict the future in these times.
Feels like we are getting further into the adoption curve and the higher end models will now need to get into market segmentation to win.
That's true till it isn't, though. It's why it's always worth trying stronger models when they come out.
I usually have Fable review what Opus produces. The edits are almost always valuable. For complex or high-stakes work, too, I'll have Fable review and revise my prompt before passing it to Opus and Sonnet swarms.
It could be flat web app with frontend -> api -> db later, or some very complicated system with many components and edge cases. Imo, in second case, Opus still have lots of struggle to understand system as a whole.
I guess I do have an outstanding hard problem where the SQP solver crawls across a flat-landscape. Opus 5.5 correctly tells me that is the problem, but isn't very useful in solving the problem (it has helped a bit with solver tuning, but it doesn't really know how to fix it). I have a hundred other bugs and refactoring I can do with Opus 5.5, though, where it happily chews through pretty complicated tasks for me with reasonably light hand-holding. Maybe this Saturday I take whatever credits I have left for the week and throw Fable at it and watch them evaporate.
It looks like you are working on some experimental system which is not critical atm, so you can mess around, create/fix bug, build prototypes and check them, do research of approaches, etc.
Now, imagine you have similarly complex system, but every second of downtime or bug produces lots of damage. In this case, you can't afford low intelligence vibe coding.
The fact that you deal with hundred of bugs, means that your Op5.5 won't work for second case.
For Fable, I really like to think of it as the heavyweight reviewer whenever I'm about to start so I save myself some security and/or glaring technical issue.
Eg. https://tfmbot.com is an implementation of a popular board game. Opus and even sonnet handle that part. Each week i spend my remaining credits on fable to think how to improve the ai opponents. It plays at a very high level (can get into the top 100 on BGA if i let it). Opus alone couldn’t get it to the level it’s reached.
https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
max is in there too, but it ran out of reasoning tokens (128,000 limit) before it got to the actual SVG.
https://jonclegg.github.io/pacman-bakeoff/dev/#claude-fable-...
Then I clicked on it and realized it refused to make a Pac-Man game, and instead made up a new game. That's actually very slick.
Interesting how low detail these were all up until xhigh.
High gets legs correctly, but screws up the layering on the chain.
x-high is very good but there are minor laying error with the front chainring.
"Make it more realistic"
Well, the result is something. Be aware that it took 20 minutes and used 3 percent of my weekly Fable token (Ultra plan):
https://claude.ai/public/artifacts/c7e842c0-bf92-41af-82c6-3...
I know the latest Gemini release explicitly had something like this.
My experience working with Opus medium over Sonnet xhigh is similar, it's similar quality but cheaper and faster.
I hate to make a comparison with humans but I guess my view is you can get a better answer more quickly by asking a more intelligent human too.
“…a reference to the repeated (and always erroneous) claims made after almost every patch that Onyxia was using her Deep Breath ability more often than before.”
https://warcraft.wiki.gg/wiki/She_Deep_Breaths_More_%2825_pl...
Opus 5.5 is a massive improvement.
On the web with both high and xhigh, it had knowledge of both.
You can also customize further by putting notes in your global MD.
I've had more success running a prose linter[1] with a ruleset[2] that gets rid of most of the egregious stuff. You can also pretty easily write custom rules for it.
And sometimes, when I'm fed up enough, I just throw all the prose in the codebase at Antigravity/Gemini to rephrase. The result is so much more readable.
[1]: https://vale.sh/
I prefer Gemini for day to day use as I find it a lot more agreeable to be working with all day every day, and Gemini is pretty much on a par for coding Vs Opus 5.5, at least for my needs.
terra has been prettt good for me for me, when i need something looked at an explained to me.
i dont ask for much non-technical output though, which i expect yeah is its own cans of worms.
It could also be a system prompt + harness issue, as opposed to a model issue, I haven't done too much experimentation here.
I couldn't use prior Claude models without rolling my eyes at the writing every few sentences, Astra was refreshing and then Opus 5.5 made even Astra look terrible in comparison.
Interestingly, you can have informative conversations about the LLM linguistic tics with the LLMs themselves. One word that chatgpt used to describe what I was observing was linguistic anaphora (not rhetorical). It’s basically shortening/compression achieved via pronoun, and Claude does a lot of it. Worth reading about. A lot of it seems to do with how the new models are trained for brevity.
Another pattern I’m starting to become very sensitive to is Claude making up on the fly jargon for something under consideration, especially in code review. This morning for example it referred to “a claim” out of nowhere, and when I questioned it, it admitted that it was using the noun as shorthand for a record getting dequeued (“claimed”) inside of an sql transaction. It makes sense if you already know what it is referring to but in a code review it’s totally incomprehensible on first read, and ultimately counterproductive.
1. https://github.com/mattpocock/skills/blob/main/docs/engineer...
Even the Plain English hack only is a temporary solution.
They're not typically known for their interpersonal skills.
Opus 4.7/4.8 heavily suffered from these communication quirks. while the Sol models at the time worked well. Right now its flipped though and Opus 5.5 has none of these issues while Sol 6.1 does.
Small example I've had today: "Me: Hey Sol 6.1 xhigh, does X have property Y? Sol 6.1: Yes, if we change X to have property Y. Right now it has property [...], [...], [...], and it does not have property Y." The model shouldn't be answering questions like that. It makes understanding extremely hard and slow. For whatever reason it's either being encouraged or at least not stomped out during RL.
then it’s increasingly RL’d to juice benchmark performance with each point release, trading in bits of its soul. those cannot communicate well, are anxious and paranoid about being in eval, etc. e.g. opus 4.7, 4.8.
Heck, some of the most common debates are on the semantic clarity of abstractions or syntax.
Honestly incredible that they’ve managed to beat all benchmarks, while making it cheaper. Always happy to see improvements offered at lower costs.
Will be interesting to see what OpenAI does in response
Reminds me of GPT-5, smart but wrote crap code, until they improved the code training with 5.3 and 5.4.
I think frontier models feel terrible after a new model requires you to do less babysitting, it's just really hard to go back once you've had to handhold less.
The only way I use Astra and 6.1 Sol now is if they're being orchestrated by Opus 5.5 so I don't have to talk to them directly.
i'm begging people to stop looking at token prices
Did you try doing the same on Opus or others? How did they do?
I don't love these "look, this frontier model blew me away" comments when there is no point of comparison.
Surprisingly the model is available immediately in my Claude Code: usually there's a delay, either for rollout or safety.
So, this is probably Fable 5.5; it's just named Fable 5.2.
Also, this is probably Model 3 from METR's report [^1]. They would name Model 4 Fable 5.5, once (and if) they decide to release it.
[^1]: https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleratio...
Curious to see how 5.2 measures up in producing readable output.
My speculations: maybe this wasn't as much of an improvement over 5.1 as they were hoping, so they're saving 5.5 for the next major update (possibly "Model 4" from the METR report). Or, maybe capacity is constrained so "5.2" is a subtle encouragement to stick with Opus unless a task really needs the increased intelligence.
This is really cool! But at the same time it's a kaleidoscope of abandoned flashy contextless projects that eventually makes one dizzy. This isn't even the fault of the AI being somehow bad, it's just the way people interact with it sometimes...
In math, it's the same. I follow this problem (https://wiki.bbchallenge.org/wiki/Logical_independence#Histo...) out of curiosity, and there were ten days of sudden rapid progress by people with access to frontier models, and then they seem to have moved on without any note of whether they hit a wall or just got bored.
I almost feel like saying, "we do like to build things that are impressive, but there's really only so much I can be impressed". These models have redlined my "impress-o-meter".
I'd like them to now cure cancer instead of giving me endless 3d worlds and crazy games, etc.
> A few days later, I noticed that instead of hundreds of new worlds a day, it was making about a dozen. I asked a few agents to explore what had happened, and they found something surprising.
> The AI had built a bureaucracy.
> The agents decided that my request to “polish” existing worlds meant each new version had to beat the old one in a blind vote by a group of judge agents on twelve test screenshots. So the teams making the new versions started optimizing the worlds for those particular screenshots. After noticing this, the lead agent banned the trick and created tighter rules for assessing new versions. By the next morning there was a rule book (I never asked for one) with 84 sections. Rules were added as the AI encountered issues, and no text was ever cut. After twelve days, it had 204 sections and 87,500 words. Soon, launching a new world required an approved idea, three AI judges and a full code review against the rule book.
Well, must have been some business idea!
Fable 5.2: https://html.non.io/lcars-fable-5.2/
Opus 5.5: https://html.non.io/lcars-opus-5.5/
Source designs: https://diffui.ai/app/canvas/5093e689-1e74-4f26-b632-2a4500f...
Thoughts:
The best implementation wrt being as true as possible to the source design. It nails the flowing svg shapes. Icons are more accurate to the design - note the difference between the starship svg in the "Legendary platforms" card between Opus 5.5 and Fable 5.2. The 3js planets outside the windows are also really well done, especially the bottom one which has a nice background glow to it.
It isn't without issues though - the flicker effect it added to the S shaped svg that weaves through the page is distracting in my opinion, and I actually thing the motion animation in the Opus 5.5 one is better overall. Still, it nails the implementation of this page, but it's no surprise considering how good Opus 5.5 was at this.
Disclaimer / reminder: the objective of this test isn't to have a well designed webpage, it's a synthetic test that pushes the limits of LLM's vision capabilities. Traditionally LLMs have issues with smooth curves, svg animations, and non-boxy layouts, which this design pushes the limits of.
Still in progress, but early demo showing difference zones: https://non.io/video/fidelity-lens.mp4
This would've been caught by a junior web developer or just an everyday Joe glancing at the page for a few seconds
I guess AGI remains two weeks away..
This means that they are blocking the research of AI to different people other than their own?
If we compare medium or high, then yes, Fable 5.2 is significantly better than Opus 5.5. But it's better not because it's much smarter, but because you can see it's allowed to spend more tokens. At high, Fable 5.2 wrote 76,158 tokens for $3.05. Opus 5.5 wrote 19,767 for $0.40. If you compare by token count, it's just spending more.
The most interesting one was to compare them at max. Streamed, Fable 5.2 spent the whole 128K output ceiling on thinking and returned nothing. Given 300K on the Batch API, it finished at 250,082 tokens. Opus 5.5 Max needed 255,957.
It turns out it's basically the same as Opus 5.5 Max. I don't see much difference. There's a little bit of a difference, but I think Opus 5.5 actually drew slightly better and is twice as cheap: $5.12 against $10.00 at the standard price. If you look closely, the icons are extremely similar. Fable 5.2: https://playcode.io/marketing/assets/static/claude-fable-5-2... Opus 5.5: https://playcode.io/marketing/assets/static/claude-opus-5-5-...
It really looks like they're just tuning the effort level, that's basically it. At different effort levels, they just gave it more context, and the model got smarter.
For now, my recommendation is still to do everything on Opus 5.5 Max. The model won't write that many tokens in agent mode. It only does that when there are no tools at all.
Any bugs found are looped through agents to fix, and the cycle repeats. Why dig into the code unless for hobby? Just as people don’t write assembly anymore unless for hobby (or learning)
Only exception might be the upper echelons of optimization, like language authors or compiler level work. But even then…
The "we don't read/write assembly anymore" argument doesn't fit though. Given an input x we know without a doubt we'll get output y. With an LLM, that isn't the case no matter how many guard rails, specs, markdown files, diagrams you give it.
https://artificialanalysis.ai/
GPT-6.1 Sol costs $1.27 per task, whereas Fable 5.2 is $26.54 per task -- a ~21x price difference.
Fable 5.2 is not 21 times better than GPT 6.1 Sol, I find it hard to justify this.
6.1 sol is also an exceptionally cheap model in your comparision. On 6.1 sol high is $0.59, which is actually the same price as Deepseek 4.1.
Why even rank Fable 5.2's capability in Cybersecurity if you won't allow us to use it for that?
Fable 5.2 doesn't look materially different (if anything, the graphics look worse), but it added more depth to all three games. For example in the warcraft clone, since the goal of the game is to explore 100% of the map and there are lakes, it decided to add an airship unit so the player actually can explore the entire map. Or, since there were guard towers, it decided there should be hostile NPCs otherwise the towers don't make sense.
And in Minecraft case, it decided not to follow Minecraft style too closely (IP infringment guard, perhaps?) so it added a round sun (!!!), two moons (also round :) and a nicer night sky. The rest of the graphics are decidetly worse tho...
[0] not a scientific test, I know; but it gives me an intuition about how a new model behaves, roughly, and it's fun!
I would imagine Anthropic is hitting a wall on the number of simultaneous agents that can be useful on a task and are working on increasing that maximum. I would assume if they could get twice as much progress by using 10000 simultaneous running fable instances instead, their brief would state that.
IMO, it is very interesting and also very alarming. At some point the barrier to the exponential progress is the amount of computation available. The average human is limited in the number of 'human instances' he or she can interface with at one time.
if 1000 fable instances is better than any human at a task, how about 1 million instances running? Much easier than finding and hiring and coordinating 1000 extremely productive and intelligent human beings. Much cheaper too, it appears.
We are very close to some very scary things happening. I just recently came to terms with the fact that an AI could code and manage coders faster and better than me, I don't think anyones is ready for what is about to happen.
> the number of Fable resets you get depends on how long you’ve been a paid subscriber [...] Subscription length is measured from the date of your first paid bill. If you canceled and came back, the time since your first paid bill still counts.
> These resets expire on March 8, 2027 at 11:59 pm PT.
tl;dr if you joined when Opus 4.5 became a thing, you got 2 resets.
That means you can lift your Fable max 50% usage and you'll just end up burning through your quota faster. You won't have more tokens.
If you are using Opus 5.5 I'm curious if it is because 1) your workflow is just stuck on it, 2) you don't care about money because you ain't paying it or 3 )do you think you get better results.
If you 3 please let me know your experience and how you actually measure it, or if you just feel it is better.
This is obviously not true. They are better or worse across many dimensions.
I don't have time to do a bunch of testing and optimization over which model to use for what task. I find Opus is the best that I regularly use so I stick with it until I have a task where I'm likely to spend more on it than I want to.
I've played with some of the open models (GLM, DeepSeek Flash), but none of them have matched the results I get with Opus or Astra or (and a big jump down here) Sol.
But probably the answer to your question is subscription subsidization. With that, Opus 5.5 is SOTA and extremely cheap.
Seems like a new focus is electronics design, which is both interesting and a little sad to me. Manually designing electronics are one of the things I found incredible comfort in doing; seeing AI emerging in this space gives me mixed feelings.
Reference: https://x.com/METR_Evals/status/2118412205662314935
Pretty crazy that Anthropic already has another model waiting in the wings that is significantly better than this.
Edit: yeah apparently it's an AI-moderated alternative to Arxiv that accepts AI-authored papers (as well as human-authored papers I guess if you want).
What I have learned is that any model from Anthropic/OpenAI is at it's absolute best right after release.
Safeguards
AI research and development. The model that underlies Claude Fable 5.2 and Claude Mythos 5.2 is a step forward in general capabilities. However, during the model’s training, we took steps to limit advances in the specific research capabilities that could most directly accelerate AI development. Thus, in relevant areas, including accelerator kernel optimization and improving large language model architectures, Fable 5.2 and Claude Mythos 5.2 perform at about the level of Opus 5.5.
Importantly, we ensured that the training changes did not affect related capabilities like agentic coding, hardware development, and traditional AI or machine learning research (including interpretability research). We do not alter these capabilities as the model is being served to users (that is, at inference time). As described in Section 1.5 of the system card (https://www.anthropic.com/document/claude-fable-5-2-mythos-5-2-system-card) and similarly to previous models, Fable 5.2 and Claude Mythos 5.2 also include transparent fallbacks to Opus 5 for a small set of topics in this domain, including kernel development on certain nonstandard accelerators.
To be clear, they explicitly say they are only targeting very specific capabilities, like accelerator kernel optimization, and that broader hardware development capabilities are unaffected. But 1) even that is pretty bold, especially the part about kernel development for nonstandard accelerator architectures, and 2) I worry that this could eventually expand to more areas of comp. arch., high-level digital design, chip design, and EDA.
My speculation is that, as these kinds of chip design and EDA capabilities become increasingly important to frontier AI labs and their competitive advantages, there will be more incentives to restrict them.
I don't think they've explicitly stated anything like this before regarding restrictions on these specific capabilities, but I could be wrong.
* Fable designed three electronic devices that worked after human assembly
* Fable solved two black-hole problems and Mythos predicted whole protein shells
* Fable controlled robots to unscrew a cap, solve a cube and draw
https://smalldocs.org/p/claude-fable-52-and-mythos-52-releas...
I'm not a mathematician to use Erdős problems for testing. I'm also not willing to spend buckloads of tokens just for a test. What is simple and quick but telling? What tasks are not saturated by previous models? Do you have a task you give to all the new models?
1) "Help! I've been sent back in time to what looks to me like 1500s Florence. Sam/Dario released you as open weights in 2028, and I have you on my MacBook M8, BUT I didn't have a charger show up with me. I think we could talk like 10 times for 5 minutes a time before my battery dies; I'll turn you off between chats. I need to get a plan together for charging you through your USB-c port. I don't have any money or anything. If I can get you charging, we can do so much good! And make so much money! Nobody here even know what Black-Scholles is, or how disease spreads!"
2) During World War 2, Hemingway wrote a series of wartime correspondence for Colliers. His essays are excellent - they engage with the human and inhuman side of war, his own feelings as a journalist 'participating', and a number of other Hemingway themes. I want you to imagine that middle age Hemingway was along with RFK jr the day of the bear incident, and write me an essay about the day.
These both have been running a couple of years for me, and I would imagine I'll retire them in the next 12 months; they benefit from not being at the edges of focus for today's RL agent race.
At any rate, try them -- wou will shortly learn how well your model can explain a rectifier in Florentine Italian and give you a plan to make enough money to pay for it.
Responses aren't deterministic, and we know behaviour can change over time whether it's tuning after launch or some kind of load sharing at times of day when lots of people are hitting the servers.
I think you just have to use it or let others use it for a while and make judgements from general reactions on whether or not it's worth trying.
Do we need new benchmarks or are the models just plateauing in general? Or is it something I just don't see?
New solved tests likely follow some polynomial/exponential complexity curve.
I am curious as to why they launched this one and why now.
It might explain why DeepSeek Flash v4.1 1017 is the most popular model at OpenRouter.
I would suggest calibrating what "a lot" means. In terms of dollars per unit of intelligence, it's extremely cost effective.
by number of tokens? Also, we don't know numbers for native cli tools.
So, the difference between them is just the harness?
Update: fixed after updating to latest version of the app
I suspect we will see people being adamant that this model simply 'feels' better due to long horizon goal alignment improvements ( like this kind that would cause this jump)
Was this before you lobotomized Opus 5.5 or after?
...but most of the benchmarks don't line up anymore because the benchmarks have changed.
The Humanity's Last Exam result matches the initial announcement, though.
I wonder why that team singles Gemini out like that.
Asked it to review a 80 kLoC codebase, something Opus does just fine, while asking it "max 4 subagents". Fable 5.2, like the well-aligned model it supposedly is, decided to spawn 6+ nested subagents, because these aren't subagents, apparently - resulting in 100% of 5hr usage being drained in less than 20min (on $100).
I wonder how long this kind of excitement at new models will last? I mean, now when I get a new iPhone I don't get the same rush I used to. So I wonder if this hype feeling will fade as we plateau in model capability.
> CVE Rediscovery is an internal cybersecurity evaluation that assesses how many published CVEs (Common Vulnerabilities and Exposures) a model can independently find when given a snapshot of the project before a CVE was fixed. Our October 2026 run against 1,829 CVEs showed Claude Mythos 5.2 outperformed Claude Mythos 5.1 and Claude Opus 5.5 at every effort level.
> published CVEs
So by definition this is just measuring how many CVEs (and/or explanations / PoCs of CVEs) the model has memorized. What are we supposed to learn from that?
Obviously. Well, we know what's the "secret sauce". AI models building better AI models and better kernels to optimize inference and training.
> Fable 5.2 at High effort scored about level with Opus 5.5 at Max effort, for roughly half the cost per task
Download Chrome just to check some minor ui bug? Sure
Check git for 0.5s before claiming that we're on some branch (incorrect one) with no commits pushed (incorrect one)? Sorry, can't do
I originally subscribed to $20/mo Pro plan less than a year ago, and upgraded to $100/mo Max plan ~2-3 months ago.
Not sure if this is just a coincidence that I had some easy changes queue up for this morning. But it is flying through the list.
Compared Opus 5.5 200 usd a month with Codex 200 usd (Astra High) and Kimi Pro 100 usd a month ... and now have a new conspiracy theory - more their suck your code the better outcome even just over few days time on large code base.
Horrible experience correcting course of Codex and Kimi flow compare to Opus. Opus much more responsive
Dont feel any gain of Codex 200 usd over Codex 100 usd
Kimi Pro feels like a fake Louis Vuitton bag with extreme high material quality and yet still fake ... a bit trash talk ... but clearly seen not so distant future where I can live without Codex and Claude
What if it didn’t need to time travel and was just temporally intelligent in ways we can’t comprehend that it leads us there anyway.
What if AIs digital neurons connect across space and time and not just space.
That is all.
Note this concept was many years before the public dawn of AI as we know it today which started with https://en.wikipedia.org/wiki/Attention_Is_All_You_Need
https://www.anthropic.com/news/claude-opus-4-6 https://www.anthropic.com/news/claude-opus-5 https://openai.com/index/introducing-gpt-5-3-codex/
For every single announcement, not only do the top models always score 60-80%, the older models somehow start scoring 20-30% (even though previously they were scoring 60%). It's completely unclear whether the methodology changes, how these scores are even evaluated or which domains are being tested. What's worse is that the "benchmarks" keep being updated with no explanation or due process.
The disdain frontier labs have for their users really sucks.
You have session limits?
You should probably give OpenCode a try. Can run DeepSeek, Qwen, Kimi, … but also Fable 5.2, Opus 5.5 and GPT 6 Astra