Benchmarking retrieval for agents on messy real-world company knowledge(www.kapa.ai)
23 pointsby emil_sorensen7 hours ago4 comments
  • emil_sorensen3 hours ago
    OP/founder of kapa (YCS23) here. Happy to answer any questions :)
    • polotics2 hours ago
      ok I bite!

      What is your opinion of:

      - BEAM

      - MemEval

      - MemoryArena

      - any other you can find on eg. HuggingFace

      How much value do you see in Karpathy's gist on the Episodic/Procedural/Semantic split (aka. btw. Doxa/Koine/Gnosis) If you do see value what parts of your design matches?

      As you are using real company data, will you at least make one step towards reproducibility by publishing some extraction pipeline so other companies can run their own comparison?

  • suhas_rnd3 hours ago
    [flagged]
  • aidiveyt2 hours ago
    [flagged]
  • haukebri2 hours ago
    [flagged]