> But GPT also offers a $100 plan with five times the usage limits. And how long would that last me?
Sol usage is basically unlimited on the $100 plan if you're human-in-the-loop, reviewing the code, and not doing some useless Gas Town crime against humanity.
What are some uses for which you found Luna to be unusable garbage?
From my side, I use Harbor to score tasks and Luna certainly performs when the prompt is either well or overspecified.
My biggest complaint is that it melds IO concerns such as timeout, with business requirements. Normally, I just to put that in the IO layer. After that, it’s basically done.
Given my pattern, a new feature normally involves Luna creating 20 to 50 files. After it thinks it’s done with its work, a Sol agent will review the code. This loop goes twice. If there’s still issues, I have to deal with them. I’ve not had to deal with them.
This when I’m at work. At home I do pretty much this but with GLM 5.3 Flash. Even less money.
If it's used for that then using anything more powerful is just inefficient usage of your tokens.
Luna medium effort is very slightly better than its output and completes tasks in 5-10x less time.
for myself anything over medium overthinks and is worse.
Even scarier is something like a legacy change made for a specific client, like “our biggest client needs checkboxes as the UI, do not switch it to a dropdown.” was in a commit message 2 years ago, AI will just overwrite that without thinking.
My AI agent never has to work with more than 5k lines of code and like 20 files at a time; and most are much smaller than that; like 1k or 2k lines... Even though one of my projects is almost 30k lines, my AI agent never has to consider more than 5k.
It's a very different experience in my day job because the project has more lines, more files and all the files for the entire project are in the same repo and naming is not as clear. In this case, I have to point to the relevant files/context in every session.
I’m talking about deterministic checkpoints and hooks. In every environment in which I need to use a coding agent I have had to essentially build a harness outside of the harness to be measurably productive. If you’re not using deterministic tools to box in what the agents are capable of doing, the problems you describe are ten times more rampant than anyone wants to admit.
I follow a specific process with agents, and it works reasonably well: 1) describe my intended feature or change and then say “please recapitulate my intention and write an implementation plan” 2) read the plan carefully and refine it, make any corrections 3) tell the agent to go, but I check in frequently as it’s going. I watch the stream of thought and make course corrections or pull it out of rabbit holes 4) carefully review the result. Checking that tests actually fail, removing unnecessary changes 5) add instructions in AGENTS.md if I’ve found a new behavior that needs redirection
I understood there were still people asking this 12 months ago. But still?
You could only possibly be doubting the time savings if you simply haven't used agentic development, whether Claude Code or other.
You have to understand, by now you basically sound like someone asking how much time you could be saving by writing in Python, instead of just jumping right in and coding assembly yourself.
If you're still asking the question, you just need to bite the bullet and try it.
If it's the kind of thing that requires a lot of iterative back-and-forth, like tweaking UX design, then there may be little speedup.
If it's the kind of thing where you there is a clear pass/fail, e.g. writing software to satisfy a comprehensive test suite that is black and white, then it could be 1000x.
The better-defined the task is and the more the agent can evaluate its own success, the bigger the multiplier.
Also, obviously, the more you are working in an area that is more unfamiliar to yourself, the bigger the multiplier.
That's why one morning I'll feel like it's maybe just 2x as it's constantly testing an app in an iOS simulator to see if a UX bug got fixed. That afternoon it might be 10x as it works on some pure code fixes that I still need to verify manually if they work. The next morning it might be something like 200x as I have an incredibly well-defined task that requires a lot of algorithmic research and 10,000 lines of code, but where the end result is easily provably correct or not. That afternoon it could be essentially "inifinity X" as it fixes a problem in a programming language I'm not even familiar with, in a domain it would take me months to get up-to-speed on, in a library I don't know, which solves a bug that is blocking me. Because it's just not something I could have realistically done at all.
So it's incredibly task-dependent. But it's pretty much always faster to some degree. Sometimes a little, sometimes a whole lot, sometimes so much faster it's still a little hard to believe.
My summary of the article: I use a really cheap LLM, and suffer like everyone else using GPT 3.1. This is definitely a throwback to the days not so long ago when LLMs were completely incompetent.
What you want is to branch questions in different agents and this is hard UX.
— Why is this function here?
This question can be but ideally should not be sent to an ongoing chat about a codebase. With some previous context, this question should become a separate "branch"/"fork"/"node" - whatever you call it. You can start a new session, share some context from previous chat and ask the question.
The reason is, and forgive me for assuming, if you are in a code/build flow, the LLM may go ahead and take action because it has a lot of previous chats with instructions to edit files.
In my experience the best way to not have these issues is to branch a new session with some context from which file you were looking at or your previous questions but as a read-only investigation.
Either this is hyperbole or something is wrong. I tried 6.1 Sol for a significant chunk of non-critical work this week and did not encounter usage depletion like this. I ran several tests that went upward of 450K tokens without depleting the 5-hour window, though I didn’t pay attention to how much.
Either this person is exaggerating, or something is extremely wrong with their harness.
I’m equally confused that their solution was to switch to Luna and try to use the small and fast model for big coding tasks. They could have switched to Claude $20/month and had ample Opus usage right new.
Most of the complaints in this article could have been avoided by not using Luna and not using a configuration that’s blowing their quota out for unclear reasons.
It’s worth taking the time to setup a style guide for your agent to remove llm-isms like this. Somewhat paradoxically, you can even have the agent help write it. Just include a couple of snippets of “bad” phrasing from previous chats, add some commentary about why it’s bad, and what to replace it with, and put that in your system prompt. I could provide examples but the whole point is that you can tailor it to exactly how you want it. This has the added bonus of being a quick sanity check for model quality. If you start noticing llm-ism appear, then likely the model is not following your instruction’s and the thread should be discarded/you need to switch to a better model
> Okay, let’s move the rule to the top. In the next reply, right in the same chat, I get text with quotation marks again.
Or maybe just luna too dumb idk. I’ve even had good success with a style guide on Bonsai 2 27B though so I doubt luna is dumber than that
Nerves of steel? Not even close.
I think 90% of the article is the author complaining about (predictably) bad small-model behavior. The large models are much better!
I suspect the whole reason for how bad models accessed through subscriptions tied to a harness of the provider (looking at you copilot) is that microsoft in the case of copilot "builds into the harness and hence into the model" an intention; and I don't have proofs (which qualifies me to start a popular podcast) but I suspect the harness keeps pushing the model to fix an issue as quickly as possible so the service can appear as "it just bam fixed the issue".
The "sitius altius and fortius" of the technobro is what you see in the tv ad "copilot check my git and fix a bug", how stupid do you think someone has to be to believe it works like that? my instructions to model say that I HATE urgency to make changes, we are researching and until we figure out causes, and understand all the mechanism involved AND THEN AND ONLY THEN we can check what's the actual intention, what business wants... then we can talk about a fix.
Grab a harness that injects the minimum possible and let the agent be and you will see how much difference there is in the quality of the model response.