The projects need to remember things, not the agents.
When they're a blank slate coming in, I get more reliable outputs.
I just have one orchestrator/secretary/router with long-term memory, but I don't switch its harness.
Here's a link to our product page, if it's helpful: https://beacon.sh
Beacon keeps the full execution trace (prompts, reasoning, tool calls, commands) outside the agent's context and distills lessons from it. Ultimately the agent still only has access to snippets of information (via skill progressive disclousure) but it's drawn from much richer session data rather than what you remembered to note down. Also you don't have to maintain it yourself!
I don't even know how obsidian works.
And I use a harness where the agents can easily look up & search other sessions, or past sessions.
I am the founder of Substrate, a tool that allows you to see all of the memory and also delete a specific entry. Is your agent allowed to write its own workarounds in your setup?