Measuring "cost per successful PR" chose Luna 6 low as the best model(www.stet.sh)
1 pointby bisonbear6 hours ago2 comments
  • djleni6 hours ago
    I haven’t really gotten into comparing models but I wonder if I’m doing something wrong or different, because results like these don’t match my experience.

    I’m using OpenCode Go/Zen with OMP and generally DeepSeek Flash will complete a task for 10 cents or less.

    Every time I’ve used any OpenAI models, even Luna, it burned through dollars for similar complexity tasks.

    Is it the effort settings? I leave OMP on auto.

    Something else?

    • bisonbear5 hours ago
      effort makes a huge difference in cost/quality. in my measurements (https://www.stet.sh/leaderboard). Luna can definitely tend to overthink on the higher reasoning effort settings. I've actually noticed that deepseek 4.1 flash max (albeit the one before 10/17) tracks close to GPT 6 Luna max in price.

      also worth noting that Luna is much more attractive on the Codex subscription plans due to heavy subsidization there - if you're using Opencode, I feel like Deepseke likely makes more sense, since they serve it very very cheaply ($60 or so of usage for $10).

  • liaozhizhi6 hours ago
    [flagged]