51 pointsby zerop2 hours ago2 comments
  • comandillos36 minutes ago
    I feel like retrieval approaches, despite how good your ranking mechanisms are, end up biasing LLMs into what a heuristics-based system with way worse understanding of the domain says. I guess is a trade off between speed and quality, but letting the LLM explore without a bias (based on my personal experience) seems to generally provide much better results in terms of quality for the final output.
  • brrrrrm14 minutes ago
    as a recovering RAG-enthusiast, I will share a quote from a peer that has really stuck with me:

    "why not use grep?"

    • goldenarm11 minutes ago
      Grep is the fastest tool for many things, but for production RAG it cannot search with meaning, different languages or typos like hybrid vector search can. And it won't scale on terabytes of documents across distributed servers if you want to say under 1000ms.
      • dv_dt3 minutes ago
        If the data is this large then why not ask the llm to build a query