*Backends:* Currently supports Qdrant, pgvector, Weaviate, Chroma, and Pinecone. Adding more is straightforward since it's just implementing a Store interface. Let me know if I missed some good backend!
*Relevance scoring:* No LLM-as-judge — that's intentional. RagTune focuses on retrieval-layer metrics only:
- Vector similarity scores (what the DB returns) - Recall@K, MRR against your golden set - Score distribution diagnostics
The philosophy is: debug retrieval separately from generation. If your retrieval is broken, no amount of prompt engineering will fix it.
For chunk size/overlap optimization — exactly the use case! `ragtune compare --chunk-sizes 256,512,1024` lets you see the impact directly.
Happy to hear feedback if you try it!