Hybrid Search fixes RAG failures: Vector + BM25 beats pure semantic retrieval

A developer building an internal RAG knowledge base tool discovered that complaints about wrong answers stemmed from the retrieval step, not the language model. After checking prompts and model settings, they realized queries containing specific identifiers—like version numbers ('v2.3 auth flow') or document codes ('policy section 7')—had no semantic meaning for vector search to latch onto. The system retrieved topically relevant but incorrect documents, and the LLM confidently answered based on those.

The fix: hybrid search combining vector search (for fuzzy intent) with BM25 keyword search (for exact identifiers), merged using reciprocal rank fusion (RRF). This approach covers both semantic and literal queries. The developer also noted that they over-engineered by setting up Qdrant too early—ChromaDB locally or even pgvector (if already using PostgreSQL) would have sufficed. They question whether pre-filtering with metadata tags at ingest could replace hybrid search or just shift the problem.