What Is Hybrid Search? BM25 + Semantic Retrieval Explained

The Problem: Neither Search Alone Is Enough

Every search method has a failure mode, and they're mirror images of each other:

Query	BM25 (keyword)	Semantic (vector)	Who wins
"SKU AZ-4471"	✅ Exact match	❌ Rare term, weak embedding	BM25
"how do I return a broken item"	❌ No keyword overlap	✅ Matches "refund policy"	Semantic
"error code ERR_4021 fix"	✅ Matches the error string	❌ Rare identifier	BM25
"best practices for onboarding"	❌ Vague, no specific terms	✅ Conceptual match	Semantic

This isn't a corner case — it's the most common retrieval quality problem in production RAG. The answer isn't picking the "better" method; it's running both and fusing the results.

BM25: The Keyword Workhorse

BM25 (Best Matching 25) is a probabilistic retrieval algorithm from the 1980s–90s, developed by Stephen Robertson and Karen Spärck Jones. It's the backbone of Elasticsearch, OpenSearch, Solr, and Lucene — the production standard for keyword search for decades.

It scores documents by exact term matches against an inverted index, with two clever mechanisms:

Term-frequency saturation: mentioning "refund" ten times doesn't score ten times higher than mentioning it once. The score saturates, so keyword stuffing can't dominate results.
Length normalization: longer documents are penalized, so a 5,000-word document that mentions a term twice doesn't outrank a 200-word one that mentions it twice.

Two parameters matter: k1 (default 1.2 — controls term-frequency saturation) and b (default 0.75 — controls length normalization). The defaults work well for general text; short structured documents benefit from lower k1 (0.5–0.8), while research papers and contracts with meaningful term repetition benefit from raising it toward 1.5–2.0.

Where BM25 wins: exact-match queries (SKUs, error codes, version numbers, names, regulatory clause references), terms the embedding model has never seen, and high-throughput retrieval — it runs on CPU from an inverted index, no neural inference at query time.

Semantic Search: The Meaning Matcher

Semantic (dense) retrieval maps text into a high-dimensional embedding space with a neural encoder. Both the query and every document chunk become vectors, and retrieval is a nearest-neighbor search: find the chunks whose vectors are closest to the query's, using cosine similarity or dot product.

At scale, production systems use Approximate Nearest Neighbor (ANN) indexes — most commonly HNSW (Hierarchical Navigable Small World) — which trade a small recall loss for dramatically faster search. That's the technology inside Qdrant, Weaviate, Pinecone, ChromaDB, and FAISS.

Where semantic wins: paraphrases and synonyms ("get my money back" → "refund policy"), conceptual queries, natural language the way users actually type it, and multilingual matching across languages.

The Hybrid Pipeline: Three Stages

1. Dual retrieval

Query the BM25 index and the vector index in parallel. Each returns a ranked candidate list — typically top-50 to top-500. Parallel execution keeps latency roughly the same as a single retrieval.

2. Fusion

Merge the two lists into one ranking. Naively averaging scores fails — BM25 scores are unbounded while cosine similarity is bounded to [-1, 1]. RRF solves this by ignoring scores entirely and working on ranks.

3. Reranking (optional)

Take the top 50–200 fused results and score each (query, document) pair with a cross-encoder, which sees both texts jointly. Return the final top 3–5 chunks as the LLM's context.

Reciprocal Rank Fusion: The Glue

RRF comes from a 2009 SIGIR paper by Cormack, Clarke, and Buettcher (University of Waterloo). For each document, sum the reciprocal of its rank position across all result lists, dampened by a constant k:

score(d) = Σ 1 / (k + rank(d))     # summed over every result list
# k = 60 (default in Elasticsearch and most implementations)

A document ranked #1 by BM25 and #3 by the vector search scores 1/61 + 1/63 — comparable, no normalization required. Documents that appear in both lists get boosted, which is exactly the behavior you want: consensus across retrieval methods is a strong relevance signal.

It's ~10 lines of Python:

def reciprocal_rank_fusion(bm25_hits, dense_hits, k=60, top_n=10):
    scores = {}
    for rank, doc_id in enumerate(bm25_hits, start=1):
        scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    for rank, doc_id in enumerate(dense_hits, start=1):
        scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)[:top_n]

Modern vector databases like Qdrant (v1.10+) and Weaviate ship native hybrid search with built-in RRF, so you don't even need to fuse client-side.

Does It Actually Help? The Numbers

On the WANDS e-commerce retrieval benchmark (Turnbull, 2025), measured with NDCG:

Approach	NDCG score
BM25 only	0.6983
Dense vector only	0.6953
Hybrid (RRF)	0.7068
Hybrid + field boosting	0.7497

A 2026 study on financial documents (Strich et al.) found the same pattern — and a striking result: BM25 alone outperformed state-of-the-art dense retrieval on financial documents, because those documents are full of identifiers, numbers, and exact terminology that embeddings smooth over. A two-stage hybrid + reranking pipeline on the same corpus reached Recall@5 of 0.816.

📈 The practical takeaway: if your RAG uses pure vector search, adding BM25 is the single highest-impact retrieval upgrade you can make. This is exactly why production RAG products — including Lawyer Assistant, which combines BM25 with semantic search and reranking so legal citations resolve to exact clause numbers and case references — don't ship with one retrieval method.

When Hybrid Search Matters Most

Legal documents — clause numbers, statute references, case citations, and plain-language queries in the same corpus
Financial reports — tickers, identifiers, exact figures mixed with prose
Product catalogs — SKUs and model numbers alongside "which one is good for gaming"
Support knowledge bases — error codes and version numbers plus natural-language troubleshooting

If your corpus is pure conversational text with no identifiers, semantic search alone may be enough — but adding BM25 costs little and protects against the exact-match queries that will eventually arrive. Need the full RAG build? Our 30-minute RAG tutorial gets you a working pipeline, and swapping in hybrid retrieval is the natural upgrade path.

Frequently Asked Questions (FAQ)

What is the difference between BM25 and semantic search?

BM25 is keyword search: it scores documents by exact term matches using a probabilistic formula, so it excels at product codes, names, and regulatory references. Semantic search embeds text as vectors and matches by meaning, so it excels at paraphrases and conceptual queries. They fail in complementary ways, which is why hybrid search runs both.

How does Reciprocal Rank Fusion (RRF) work?

RRF merges ranked lists by summing 1/(k + rank) for each document across all lists, with k typically 60. It ignores raw scores entirely and works on rank positions alone, so no score normalization is needed. Documents that rank well in both lists get boosted.

Is hybrid search better than pure semantic search?

Yes, for most RAG systems. On the WANDS e-commerce benchmark, hybrid with RRF scored 0.7068 NDCG versus 0.6983 for BM25 alone and 0.6953 for pure vector search. Hybrid retrieval plus reranking reached 0.7497. If your RAG uses pure vector search, adding BM25 is the highest-impact retrieval upgrade available.

When should I use hybrid search?

Whenever your documents contain exact identifiers (SKUs, error codes, clause numbers, names) alongside natural language. Legal, financial, product, and support documentation are prime candidates. For pure conversational corpora with no identifiers, semantic search alone may be enough.

Do I need a cross-encoder reranker?

Not always. RRF alone already beats either retrieval method alone. A cross-encoder reranker is the final polish: it re-orders the top 50–200 fused results by jointly scoring each query-document pair, giving the best final context for the LLM. Start with hybrid + RRF, add reranking when you need the last few percent.

Can I build hybrid search with local tools?

Yes. Qdrant and Weaviate support native hybrid search with sparse (BM25) and dense vectors plus built-in RRF. ChromaDB and FAISS can serve the dense side, and libraries like rank_bm25 or Elasticsearch provide the keyword side — then you fuse client-side with a few lines of Python.