I'm honestly not sold on memory layers as a whole, I find that adding more context only serves to make the LLM's dumber and I've been slowly leaning towards just having no memory and working on features in isolation.
You've done benchmarks already so that's another step in the right direction, do you find that these benchmarks are good indications of practicality? Have you seen a major difference in LLM performance utilizing this as a memory layer?