All of these resources on agentic workflows, managing agent memory, harness engineering, etc. appear to just be theatre to me.
Those were mostly obviated by reasoning models and harness updates by 2024.
It seems pointless to invest energy into the latest/greatest AI technique or framework when they're going to either be absorbed or replaced on a 3 month cycle.
Of course, there are other areas that can improve model output (Direction rather than open-ended assistance requests, using keywords + plugins that help, the "your output should include: " style prompting).
A few of us run almost the same exact setup at my shop (Base Claude Code w/ SuperPowers + a context repository) and the models are somewhat unhelpful to some, and give meaningful output to others. The only correlation I notice is that their prompts are no-good. Not from a meta "prompt" engineering standpoint, but from a general English 101 standpoint.
"dudde no i wanted the function to return 3 things. not like that. do it again"
VS something like
"Modify the "renderThreeVars()" function signature to accept another variable called "z" and add it to the return statement at line 64."
Obvious exaggeration, but you get the point.
I ask it all the time about whether X is feasible, how we can get started on Y, and to investigate issue Z.
It is working great for me in a >100k LOC project.
Perhaps this works less well with weaker models. I suspect the people who say Qwen 3.6 27B is working well, are using prompts like "modify the renderThreeVars() function in rendering.py".
The prompts were, predictably, really bad. Broken English, sentence fragments, vague requests, lack of context. Yet somehow, the users always got the answer they were looking for. It might have taken a few extra turns with questions from the model, but the end result was the same.
It's humbling, but a flowery, carefully crafted prompt is at best slightly more efficient than a "CAN A DOG BE EATIN SUN FLOWER SEED?" peasant prompt.
Unless the user wanted to know if a cat could eat sunflower seed or something.
ChatGPT kept the charade for all of one sentence. Then it dropped to talking about "your dog" the rest of the way. It even starts the final paragraph with "if, instead, you mean you (a human) ate them...", and finishes with the question "is this about an actual dog or yourself?"
Google did consistently refer to me as a dog, but its entire focus was on the steps "your human" should take, no advice for the dog itself.
In both cases, it looks like the first person roleplay was entirely inconsequential for the usefulness of the output. I think the current gen AIs have outgrown this trick and you can safely forget about it.
Why is would that be "bad prompt"? It is machine inpit, if machine can interpret fragment all the better.
Are people really putting on their resumes that they are capable of reading and writing and appropriately defining and limiting context? That's all prompt engineering is - it's being able to communicate effectively and elucidate your objectives.
Congratulations to all you English majors out there, you're about to make $350K/year.
People put whatever buzzwords will get them through initial screening. My resume contains tons of banal shit like agile, automated testing, Linux, AI (since 2018), and design patterns.
Yes, it is. Source: had models inplement complex things from scratch and bullshit regardless of whether it was a one-line prompt or a detailed "SOTA witchcraft magic spells that are guaranteed to work"
It's like imagining you could "prompt" Richard Feynman to be smarter at Physics.
That is, for 99.9% of engineers, if you want the model to do a code review of your project, the best solution is to just ask Fable, "Hey Fable, do a code review of this project." Throwing in extra text like "think like a senior engineer", "ensure you focus on DRY principles, KISS, self documenting code, etc", doesn't make a difference.
These sorts of tricks used to work with dumber models, but now, like I said before, it's like thinking you can prompt Linus Torvalds into writing better C than he already can do.
> it's like thinking you can prompt Linus Torvalds into writing better C++ than he already can do.
Linus Torvalds, the inventor (and beloved dictator) of Linux, has always been quite harsh about C++ and why he rejects it for Linux kernel development. He’s not just been very vocal about it, but also brought up some arguments against the use of C++ that are worth reviewing in detail.
The models are beyond expert level in many areas at this point.
Do you really believe that adding extra junk to your prompt is going to make the model write code better than it does already?
Again, imagine going to Terrence Tao and "prompting" him to get better at Maths, do you think you can do it? What prompt would you give to him to make him produce better maths. Unless you're already a world-leading Mathematician I think you would find it hard.
It's not that the models aren't smart or whatever; we know they're extremely capable.
There's always going to be value in being able to clearly communicate what you want the model to do, especially when the context is lacking.
Being able to write clear, always is.
These tools work pretty well out-of-the-box. I'm sure I could squeeze out better token usage or streamline some tool calls, but it's not something I really want to focus on. Just like I don't want to endlessly configure my IDE, I don't really have patience with spending time on anything besides actually building something.
LLMs seem terrible at using LLMs in harnesses. Have you seen how they rot their context with the stuff they put in .md files if you let them?
You'd have to have the LLMs search, and thus these resources could be for them more than you
Another thing in it is a strict line count. Any increase in line count requires my approval. That last one is important because it plays well with two biases: models don't tend to create long lines so they won't try to cheat that way, and they're strongly inclined to keep churning out lines so I take that away from them.
Unless I'm doing something super complicated even taking time to set things up like subagents, etc. seems like a waste compared to just building.
The only things that really seem beneficial (for Claude Code) seems to be learning to set up loops, memory and finding relevant MCP servers.
Just use the vanilla settings.
First you have to know that "it" exists and is possible. A cookbook like this introduces readers to concepts and features they didn't know were there in the first place.
Now process engineering is more important than prompt engineering.
I appreciate there may be more efficient ways to do things if you're an actual developer or engineer but I'm not and the tools I've been able to build so far have been both fun to make and add value to our workflow as a small 2 person ecommerce business.
I do at least have a background in web as a designer for many years and having worked with developers I can at least spec and understand how a product might work which is one thing Claude/AI isn't hugely helpful with and often it's the testing and QA phase as always where the problems and shortcomings expose themselves.
the point of guides is to provide assurance for people unfamiliar to the process in the first place
Before /goal was ralph-wiggum it was def an interesting learning experience, it was interesting to see how Claude became a lot better doing this itself but it still took 6 Month and more.
You can wait and sit it out and suddenly you get fired if you miss the point when to start spending more time and energy on topics like this.
https://platform.claude.com/cookbook/coding-prompting-for-fr...
"Use a black background and this hideously wide font!"
Personally I prefer the before shots all the way down.
I've also found that they tend to coerce the programmer into thinking about the end result, rather than try to push them out of that role. His grill skills are about making sure the actual requirements are known, and his prototype skill can help explore things that need to be experienced to make a hard decision on.
Before: bland
After: bland with gradients
Other AI labs also tend to publish examples and cookbooks on GitHub and Hugging Face, so it's always worth keeping an eye on those as well.
I have been using Magic Patterns which is very good at generating initial prototypes, especially when driven by an agent familiar with it's facilities and constraints.
I assume the service is just a wrapper plus scaffolding but I like it. You have to push it though to get anything particularly creative but it's good at standard front end designs.
https://www.magicpatterns.com/
That's not necessarily what you're after because I only use it for piloting stuff, not for adhoc feature addition and fixes.
I can't use it on good faith Same goes for grok, chat gpt and others all involved in killing people and monopolising the market.
It turns out the "average" version of a recipe that's baked into the weights is usually a solid recipe, and they're really good at offering substitutions for things like "I don't have ingredient X" or "make it vegetarian".
It's also fun promoting "make it tastier" once or twice after each recipe just to see what happens.
Works surprisingly well, can even tell it "I have so and so ingredients in my pantry, I want a keto-friendly meal, what can I make?" followed with some narrowing down dialogue-style.
I don't need to tell the agent my tech stack, what DB access library I'm using, the way in which I write my tests, etc. Why? Because it's all very clearly spelled out within the code itself. I've gone great lengths to make sure that my fairly complex domain can be understood by a person with little domain knowledge, which means it should certainly be understood by a tool that has a wealth of it.
This is to say, Anthropic seems to be going down the path you are describing to make the CLAUDE.MD unnecessary. But I think that's the Claude Code harness.
My system CLAUDE.MD spells out my preferences in stack and architecture, and how I expect the agent to interact with me.
I have it generate a summary doc in large repos so it doesn't need to slurp in the world to review a PR or work on a corner of it.
I'm not enough of a designer to be able to point exactly at what makes it so, but Claude does seem to have a somewhat limited repertoire of styles.
Maybe if you could point more precisely at the required changes, you could discourage it?
OpenAI's started in June 2022 https://github.com/openai/openai-cookbook/commit/535f545be7e...