It's same for me. I decided I want to control this somehow, at least the procedure of how memories are collected, and where. To be able to see them, process them, share them, mention them and outside of Claude storage, in my format.

I work in Cowork with a filesystem I control, or in Claude Code and also with Codex.

I instructed Claude and Codex to checkpoint into my external memory: keep a session journal in a fixed format markdown with front matter (date of creation, summary, keywords, status, links to related entries) and the body of such entry is the actual context and important notes, including also links to relevant files. (porbably what Claude or Codex could write in its own memories) and add such record to a common journal index. Notes are captured when I ask, and periodically if agent feels that we have some "milestone" in conversation. This means I now can ask Claude to make the "handoff" from such jounrals (give the filesystem links of the needed journals + the additional information) to "onboard" the other agent. I then simply place this in new chat and the agent onboards itself. I also can pick the needed articles myself and ask the agent to start from <these specific files> or let it search there ("start where I talked about.. on last week.."). Agents can search in this "context articles cloud" very efficiently. These "articles" are "memos" and are stored in one place on my file system organized by month when they were created and then by topics on which the agent was asked to work. They contain links to the project's relevant files, links to other notes in this knowledge base, and contextual information, real "context articles". Perhaps how agent would store it in their memories but now it stores in mine :) This also nice place for me to see that all, notes have clean visual organization and recognizable names. This really helps me.