Hybrid Search: Dense + Sparse (BM25)

Introduction

Welcome to Advanced Retrieval — the section where we make the "R" in RAG genuinely good. Everything so far improved what we index (chunking, context, representations). Now we improve how we search, and we start with the highest-ROI, most universally-applicable upgrade of all: hybrid search.

Here's the uncomfortable truth that motivates it: the semantic vector search you've been building this whole container — the thing everyone calls 'modern' RAG — has a serious blind spot. Ask it for a product code, an error ID, a person's name, or a rare technical term, and it often fails — losing to a keyword-matching algorithm from the 1990s.

The fix isn't to pick a side. It's to use both lenses at once. In this lesson you'll learn:

Why dense (semantic) and sparse (BM25) retrieval fail on opposite queries
How BM25 actually works — and why it's a brutally strong baseline
How to fuse them (and why you can't just average the scores)
SPLADE (learned sparse), and the honest answer to "is hybrid always worth it?"

Two Lenses, Opposite Blind Spots

There are two fundamentally different ways to decide if a document matches a query:

🧠 Dense retrieval (semantic / vector) — embed query and documents, rank by cosine similarity. It matches on meaning, so it shines at paraphrase, synonyms, and concepts: a query for "sign-in details" finds a doc about "passwords" even with zero shared words. Its blind spot: exact, rare tokens. Identifiers like E-4012, part numbers, person names, and out-of-vocabulary jargon get smoothed over — the embedding captures the general 'shape' and loses the precise string.

🔤 Sparse retrieval (lexical / BM25) — score documents by exact term overlap, weighted by how rare each term is. It's the opposite: it nails exact keywords, codes, names, and rare technical terms, because it's literally matching the string. Its blind spot: meaning. It has no idea that "car" and "automobile" are related — no shared token, no match.

This is the whole insight: dense and sparse fail on opposite queries. Where one is blind, the other is sharp. That complementarity is exactly the condition under which combining two methods helps — and it's why hybrid search is so consistently effective.

An infographic titled 'Hybrid Search — Dense + Sparse, Two Lenses on Relevance'. Two cards compare the paradigms. Dense (vector / semantic), using embeddings and cosine similarity: wins on meaning, paraphrase, synonyms, and concepts (so 'sign-in details' matches 'password'); misses exact codes, IDs, names, and rare or jargon terms because it blurs something like 'E-4012' into mush. Sparse (BM25 / lexical), using exact term overlap and an inverted index: wins on exact keywords, product codes, names, rare technical terms, and version numbers; misses synonyms and paraphrase because it has no semantics ('car' does not equal 'automobile'). They fail on opposite queries, so run both and fuse — but scores live on different scales (BM25 unbounded, cosine minus-one to one), so don't naively average; fuse by rank (RRF, the next lesson) or carefully-normalized scores. BM25 in one line: term frequency (saturated, via k1) times inverse document frequency (rarity) divided by length normalization (b); decades old, fast, interpretable, and a brutally strong baseline that pure vector search often loses to on exact-match queries. SPLADE (learned sparse): a BERT-based sparse encoder doing term weighting plus expansion (adding related terms), bridging BM25's exactness and dense's semantics, with document vectors precomputed at index time. A caveat: hybrid is not a free win — it lifts most workloads (on WANDS, 0.75 versus about 0.70 for either alone; BM25 even beats dense on code and finance docs) — but with top-tier embedding models the gain can shrink or add noise, so measure on your data. Banner: semantic meaning plus exact terms; hybrid catches what either lens alone misses — fuse the results, then rerank.

BM25 — the 1990s Algorithm You Can't Beat by Ignoring

Before fusing, respect the sparse leg. BM25 (Best Match 25) is the workhorse of lexical search — an extension of TF-IDF — and it is shockingly hard to beat. It scores a document by summing, over each query term, three intuitive factors:

# BM25 score of a document D for query Q = sum over query terms t of:
#
#                          IDF(t)  ×        TF(t, D) × (k1 + 1)
#   score(D, Q) =  Σ      ───────────────────────────────────────────────
#                  t∈Q                TF(t, D) + k1 × (1 − b + b × |D|/avgdl)
#
#   IDF(t)            rare terms count MORE (an "informativeness" weight)
#   TF(t,D)·(k1+1)/…  term frequency, but SATURATING — the 10th occurrence
#                     barely beats the 3rd (k1 controls how fast it saturates)
#   b × |D|/avgdl     LENGTH NORMALIZATION — long docs don't win just by being long
#
# Defaults k1≈1.2, b≈0.75 work well out of the box. It's an extension of TF-IDF,
# decades old — and still a brutally strong baseline.

IDF (Inverse Document Frequency) — rare terms count more. A match on "E-4012" (appears in 1 doc) is far more informative than a match on "the" (appears everywhere).
TF with saturation — more occurrences help, but with diminishing returns: the 10th mention of a term barely beats the 3rd (the k1 knob controls how fast it saturates). This stops keyword-stuffed documents from dominating.
Length normalization — long documents don't get to win just by containing more words (the b knob controls the strength).

Defaults (k1≈1.2, b≈0.75) work well out of the box, it's fast and interpretable, and a 2026 benchmark even found BM25 outperforming state-of-the-art dense retrieval on financial documents dense full of identifiers and exact terminology. The lesson: don't dismiss lexical search as 'old' — it's the exact-match leg your semantic search desperately needs.

See It: Where Each Lens Breaks

This is the entire argument in one widget. The same five documents are ranked three ways — Dense, BM25, and Hybrid — for two queries: a paraphrase and an exact code. Watch which ranker gets a ✅:

Two queries, three rankers. A paraphrase breaks BM25; an exact code breaks dense embeddings. Watch which column gets a ✅ — hybrid is the only one that wins both.

The pattern is stark and opposite:

On the paraphrase ("change my sign-in credentials"), BM25 misses the password-reset doc entirely — zero shared words — while dense finds it.
On the exact code (E-4012), dense ranks the wrong document #1 (it blurs the code), while BM25 nails it.
Hybrid is the only column with a ✅ on both — it inherits dense's semantics and BM25's precision. That's the payoff.

Fusing the Two — and Why You Can't Just Average

So you run both retrievers and combine their results. The naive instinct — add the scores — is wrong, and understanding why sets up the next lesson:

The scores live on incompatible scales. Cosine similarity is bounded in [−1, 1]; BM25 is an unbounded positive float (it can be 2 or 40 depending on corpus and query). Add them and BM25 silently dominates — its bigger numbers swamp the cosine scores, so you're really just doing BM25. Min-max normalizing first helps, but it's brittle: a single outlier BM25 score compresses every other document toward zero.

The robust, standard fix is to throw the scores away and combine the ranks instead — Reciprocal Rank Fusion (RRF). Position is a universal language every ranker speaks: being #1 means the same whether the underlying score was 0.9 or 92. RRF is the de-facto default in Weaviate, Qdrant, OpenSearch, and friends — and it gets its own lesson next, because it's the key that makes hybrid (and multi-retriever fusion generally) actually work.

(The interactive above uses careful min-max normalization to keep things visual; in production, prefer RRF.)

The good news: you rarely implement this by hand. Native hybrid is everywhere in 2026 — Weaviate (since 2022), Qdrant (one query, RRF internally), Pinecone, Elasticsearch/OpenSearch, pgvector + tsvector. (And recall Anthropic's Contextual BM25 from L94 — that was the sparse leg of a hybrid system.)

SPLADE — When Sparse Learns Some Semantics

There's a third option worth knowing, sitting between BM25 and dense: learned sparse retrieval, the best-known being SPLADE.

SPLADE uses a BERT-style model to produce a sparse vector (one weight per vocabulary term, mostly zeros — still inverted-index-friendly), but with two learned superpowers over BM25:

Term weighting — it learns which terms matter, beyond raw frequency.
Term expansion — it adds related terms that aren't in the text. A doc about "cars" can get nonzero weight on "automobile" and "vehicle" — so it captures some of the synonymy that trips up plain BM25, while keeping exact-match precision.

The catch is latency: running SPLADE on every query at search time can add 100–300ms. The standard mitigation: precompute SPLADE vectors for all documents at index time, and run the model only on the (single) query. Qdrant and others support sparse vectors natively for exactly this. Think of SPLADE as a stronger sparse leg for your hybrid system when plain BM25's lack of semantics is hurting you.

The Honest Take: Is Hybrid Always Worth It?

Hybrid search is one of the highest-ROI retrieval upgrades you can make — and for agents and production RAG over diverse content it's close to non-negotiable, because real queries are full of proper nouns, version numbers, and IDs that pure vector search fumbles. On the WANDS e-commerce benchmark, a tuned hybrid reached 0.75 NDCG vs ~0.70 for BM25 or vectors alone (~7% lift); on code- and finance-heavy corpora the gap is larger.

But — and this is the expert nuance — it is not a guaranteed win:

Some experiments found that adding BM25 helped smaller embedding models but slightly hurt top-tier ones — the strongest modern embedding models already encode a lot of lexical signal, so the extra sparse channel can be redundant or even add noise.
It adds real complexity: a second index, fusion tuning (RRF's k, or weights), and more moving parts to operate.

So: default to hybrid when your content has exact-match-critical tokens (codes, names, jargon, identifiers) — which is most real-world content — but treat it as a hypothesis to validate, not dogma. Run your own retrieval eval (recall, NDCG) with vectors-only vs hybrid on your queries before committing. As with everything in this section: measure, don't assume. And whatever you choose, the next layers — fusion (RRF) and reranking — compound on top.

🧪 Try It Yourself

Drive the widget, then reason like a retrieval engineer:

In the interactive, which ranker fails the paraphrase query, and which fails the exact-code query? Why is hybrid the only one that passes both?
You're building search over a software bug tracker — queries are a mix of error codes (NPE-238), stack-trace snippets, and natural-language descriptions ("app crashes when I log out"). Dense-only or hybrid? Why?
A teammate fuses dense + BM25 by adding the raw scores and results look like pure keyword search. What went wrong, and what's the fix?
Plain BM25 matches "laptop" but misses docs that only say "notebook computer." Which sparse upgrade closes that gap, and what's its cost?
You swap in a top-tier embedding model and your hybrid lift vanishes (even dips). Is your hybrid setup broken?

→ (1) BM25 fails the paraphrase (no shared words); dense ranks the wrong doc on the exact code (it blurs E-4012); hybrid combines both signals so the right doc surfaces either way. (2) Hybrid — error codes/stack traces need exact match (BM25), descriptions need semantics (dense); it's the textbook complementary case. (3) Raw BM25 scores are unbounded and swamp cosine's [−1,1] → it's effectively BM25-only; fuse by rank (RRF) or normalize carefully. (4) SPLADE (learned sparse with term expansion adds "notebook/computer"); cost = model latency, mitigated by precomputing doc vectors at index time. (5) Not necessarily broken — strong embedding models already encode lexical signal, so hybrid's marginal gain can shrink or add noise. Measure vectors-only vs hybrid on your eval and keep whichever wins.

Mental-Model Corrections

"Semantic (vector) search is strictly better than keyword search." No — they have opposite blind spots. Dense blurs exact codes/names/rare terms; BM25 can beat dense on identifier-heavy corpora.
"BM25 is obsolete." It's a brutally strong, fast, interpretable baseline and the exact-match leg of modern hybrid systems.
"To fuse, just add the scores." Scales differ (BM25 unbounded vs cosine [−1,1]) → BM25 dominates. Fuse by rank (RRF) or normalize carefully.
"Sparse vectors can't capture meaning." Classic BM25 can't, but SPLADE (learned sparse) adds term weighting + expansion — some semantics, still sparse.
"Hybrid always wins." Usually a big win — but with top-tier embedding models the gain can shrink or add noise, and it adds complexity. Measure on your data.
"Hybrid is exotic to build." It's native in Weaviate, Qdrant, Pinecone, OpenSearch, pgvector — often a single query flag.

Key Takeaways

Hybrid = dense (semantic) + sparse (BM25/lexical), because they fail on opposite queries: dense wins meaning/paraphrase but blurs exact codes/names/rare terms; BM25 wins exact tokens but is blind to synonyms.
BM25 = IDF (rarity) × saturating TF (k1) ÷ length-norm (b) — a decades-old, brutally strong baseline pure vector search often loses to (and sometimes beats dense outright on identifier-heavy docs).
Fuse by rank, not raw score — different scales make naive averaging collapse to BM25-only; RRF (next lesson) is the standard fix. Native hybrid is everywhere (Weaviate/Qdrant/Pinecone/OpenSearch/pgvector).
SPLADE = learned sparse (term weighting + expansion) — a smarter sparse leg that captures some semantics; precompute doc vectors at index time.
Honest: high-ROI and near-essential for agents/diverse content, but not guaranteed (can be redundant for top-tier embedders) and adds complexity — measure.
Next: Reciprocal Rank Fusion (RRF) — the elegant one-line algorithm that actually combines these (and any) ranked lists.