A verification loop 4x'd DeepSeek's intelligence, matching Opus at 1/7 the cost(ironbee.medium.com)

21 pointsby sozal3 hours ago5 comments

bel82 hours ago
From what I understood, the gist of it is that uses a bunch of skills in a loop to help LLMs solve the tasks better via browser-testing, debugging, visual-testing, etc..
The skills can be found here:
https://github.com/ironbee-ai/ironbee-devtools-skills/tree/m...
Question that was left unanswered:
How much longer does a 4x DeepSkeep loop took compared to Opus?
2 hours ago
undefined
_aavaa_3 hours ago
Seems interesting, but buried under a mountain of written slop.
- rightbyte2 hours ago
  It is marketing for some SaaS thing. However, I think something like this, "verification loops with some sort of tricks", is what Anthropic and "OpenAI" do behind the scene with their interfaces.
- sozal3 hours ago
  Fair on the length. The verification loop took DeepSeek from ~20 to ~80 on Web-Bench, matching Opus at ~7x less cost. Curious if the method holds up for you.
  - _aavaa_2 hours ago
    Well I don’t know what the method is, there’s a mention of ironbee, but I’m not willing to spend the time digging through this to figure out what’s needed and how to set it up.
    alexgoodhart2 hours ago
    You could literally send it to an ai
    scotty792 hours ago
    > I’m not willing to spend the time digging through this to figure out what’s needed
    That's the neat part. You don't... need to do this yourself nowadays. AI agent is perfectly capable to figure out any new open source thing that pops up daily now. You can just give it the link to the repo and say "Set this up for me. I wanna play with it."
3 hours ago
undefined
3 hours ago
undefined