Whatever Opus 5 is doing should not happen.

Prompt was "read and update the config file with new data". This work on 4.6 takes <2 minutes to read the file, parse the new data, and patch.

Opus 5 Result: 43 minutes of pulling containers, running sandboxes, creating testing suites, which included evaluating the entire repo beyond the scope of the config file.

Both: one file modification

Not specifically Anthropic but why are we allowing billing to take place in tokens that are nebulous and fully controlled by the operators who have no aligned incentives?

If I have a user input and then sanitize and inject that into a prompt to do something, I have no idea how much that is going to cost at all and no real way to measure this properly. A parallel example is digital ocean or aws, i can go and measure/limit my compute/fs/memory/startup times/etc and while it can be impossible to get down to the last flop of money allocated - i can run things on a real budget with real constraints, opposed to an LLM where I have to .. prerun a sanitized user prompt through a tokenizer and then ask an LLM to guess what it may do and give token consumption estimates and then act on those in any sane manner for the user?

Perhaps i'm missing something to do realistic and static rails on things but I don't see a serious way at scale to use the token billing model handling things requiring a users free text input short of having to go pander to VC money to throw money at it until someone else figures it out.

*to clarify my rambling... We should be billed and given controls based on resource usage itself and not an opaque token concept on top of not being able to spin any knobs that control it's resource usage.

LLM users don't want to put in effort, so they offload tasks to LLM.

LLM doesn't seem to be keen to put in effort either!

Is this AGI?

I suspect it's not just this, there's plenty of 'optimization' around rubberbanding usage limits as well as routing to a different model in the backend. The incentives are too strong.

undefined

What’s currently the best pattern if I want to combine Fable and GPT if a workflow but keep using subsidized tokens?

Is it actually entirely a prompt-based information? I’d assume that some of it is the harness part of the agent setting reasoning token budget and compacting reasoning etc.

In that case, the agent will respond incorrectly because it has no visibility into what reasoning mode it’s in.

Does anyone know what setting effort means for models like these? Do they allow longer thinking sessions? Some kind of system prompts? What’s stopping someone from getting max effort output from low effort setting?

I mean whatever models I use (with Claude code) sub agents seem to use absurd amounts of tokens for trivial (or at least small) tasks.

So glad I switched away from Anthropic. I'm certainly running into problems with OpenAI but nothing quite on the level of Anthropic's insufferability.

I submitted an application for Anthropic's Cyber Verification Program.

I was approved.

3 months later, my approval was degraded into "in review" (revoked). I'm sure my account was flagged based on contents of debugging/researching firmwares/etc.

I opened a support ticket. No response. I opened another support ticket. No response.

1-2 weeks later, I got a response that I will not be re-approved and I need to reapply. No problem.

The page to reapply on does not allow me to re-apply because it my account is stuck in an "in review" status.

https://github.com/anthropics/claude-code/issues/84352

The community thinks it's a bug. I'm 95% sure it's not and a bunch of us who were previously approved had it revoked due to flagged content and will not be reapproved.

I switched to Codex + got TAC approved instantly and have not looked back. It's a shame. That's 100% separate from whatever the heck the quality of Opus 5's outputs are. The way it talks... insane. I would bet a good amount of money their next release will focus "reduced simplified responses" if I had to guess.

$2t company by the way

I have been as well. Based on my own sessions, Max vs Max, same 1M context window size, the literal majority of the cost overhead of Opus vs Sonnet comes from Opus being chattier. So I started using Low reasoning instead of falling back to Sonnet, and I've been really happy with the results. Way better quality at a comparable spend. I also rarely go past High lately, which was another major cost save.

[flagged]

...the evidence, as best I can tell from the tweet, is that they asked Claude what effort level it was set to. But how would the model even know that?

Not convinced here.