This feels like a kid trying to do science. The will is there, but lacks experience.
It's funny, when I saw the title I was hoping the article would include some sort of blind ranking, where you could see the outputs (without knowing which model they came from) and score them on some criteria. Could have been a fun way to get a better ranking of the results.
I keep it pretty up to date (tomorrow Grok 4.5 and Sonnet 5 should be pushed).
Though it crunched most of the free quota, 47111 tokens, so I couldn't make multiple attempts.
I really wish they'd thrown in something like GLM-5.2 into the comparison.
We made Grok 4.5, GPT-5.5, and Claude write a blog post about using Grok 4.5, GPT-5.5, and Claude to build the same apps.
I sort of do this whenever an interesting new model comes out.
I am trying to figure out how many LLM converged on a writing style that resembles a LinkedIn MBA true believer. Maybe because there was just such a sheer mass of corporate-speak drone writing out there in the wild in the training data set?
But more seriously, is there a firefox extension that 'skims' the text body content of a page and puts some kind of "this was probably written by AI" meter, gauge, number or indicator in the top menu bar adjacent to the URL bar? It could even be color coded in various shades from green, yellow, orange, red. If there isn't, it sure seems like something that would be good to have.
> “snappy stylist”
Funny you can tell its slop just by this
I don't get why cost per reply is at all relevant here?
Why do so few who attempt comparisons actually compare dollars per task.
Yes, subjective. But it matches my repeated experiences with these models for what it is worth.
Do it again but this time get them to make a multiplayer online Jetmen REVIVAL game. Online play is key, because it's very complex. Jetmen is a good game for this since it has physics and customization that's complex enough but still simple.
> "Nay, laddie, that’s no’ the real AI Scotsman! He’s grander still! More powerful! Just wait for the next model!"
What’s more interesting to me than time-to-first token or latency is the time it takes for the agent to execute, from starts to finish, excluding when it’s waiting on a human.
On which note I recently convinced codex to use the ChatGPT web client to run subagents. Means I only pay for the slavemaster, and all the slaves I can eat for $20 a month. Actually works surprisingly well - I currently have it crunching through a large dataset, which would have taken weeks on a single thread - started last night, nearly done this morning. $20.
GLM is the clear winner:
I use different models all the time. And mostly lower cost ones. I do not know how people write software these days, but I have clean instructions, usually in Epics and they have Tasks.
I have been using DeepSeek V4 Flash for much of my coding in https://github.com/brainless/akar for example. Planning is mostly done by Qwen latest (in opencode) or Sonnet.
For my commercial, client work I use Claude but barely use Opus. Sonnet does most of the work. For a recent project, I went through a 35 page PRD in about 4 weeks, that includes client calls, changes, Ecpi/Task generation, a massive test suite, deployment.
Variance in quality on these things is so, so high.
Written by Claude. Ugh. If it’s worth publishing, it’s worth proofreading, folks.
For hard tasks , that needs precision I will wait and pay expensive tokens
For everything else , query data , logs, rolling out releases , I’m using grok and it’s much better vs other tools and much cheaper too .