Why is dense retrieval on its own not enough, and how do you combine it with keyword search?

The two retrievers fail on different queries

BM25 scores a document by how often the query's terms appear in it, discounted by how common each term is across the corpus and dampened so that repetition saturates rather than accumulating without limit. Everything it knows about meaning is the token. That makes it exact and unbeatable on the queries where the literal string is the point — a part number, an error code, a person's surname, a function name, a rare piece of internal jargon — and it makes rare terms powerful, because inverse document frequency gives a term that appears in three documents enormous weight. It makes BM25 helpless on vocabulary mismatch: a query about "cancelling my subscription" against a document that only ever says "terminating an agreement" scores near zero.

Dense retrieval is the mirror image. It handles the paraphrase, the synonym and the question asked in the user's words rather than the author's, and it dissolves exactly the tokens BM25 relies on, because an identifier is split into meaningless fragments whose vector is dominated by surrounding context. Two distinct part numbers in similar sentences end up close together.

So the argument for hybrid retrieval is not that two signals are generally better than one. It is that the union of the two covers query classes that neither covers alone, and that the classes are predictable: literal, rare and identifier-like queries go to the lexical side, vague and paraphrased ones to the dense side. A real query log contains both, usually in a proportion that surprises whoever assumed users would write well-formed questions.

Scores from two engines do not share a scale

The awkward part of combining them is that the numbers are not commensurable. Cosine similarity is bounded, and in practice occupies a narrow band, because unrelated text is rarely orthogonal. BM25 has no upper bound at all: its magnitude depends on the number of query terms, their document frequencies in this corpus, and the length of the document. A BM25 score of 18 tells you nothing without knowing the query it came from.

The obvious repair — min-max scale each list to the unit interval and add them — is worse than it looks, because the extremes it scales by come from the candidate set rather than from the corpus. The top result of every list becomes 1.0 whether it is superb or barely relevant, and the bottom becomes 0.0 whether it is rubbish or nearly as good as the top. That means the mapping changes per query in a way unrelated to quality, so a strong lexical match on one query and a weak one on another are assigned the same fused weight. It also makes the fusion sensitive to how deep each list is: extending one retriever from 20 to 100 candidates changes the normalisation of the entire list and therefore reorders the top of the fused result, which is a deeply confusing thing to debug.

Z-score standardisation is a modest improvement because it uses the distribution rather than the extremes, and calibrating each retriever's scores against a held-out set is better still. But both need per-corpus work, and both go stale when the corpus or the embedding model changes.

Fuse on rank instead

Reciprocal rank fusion sidesteps the problem by discarding the scores and keeping only the ordering each retriever produced. Each document scores the sum, over the lists it appears in, of one divided by a small constant plus its rank in that list — the constant, conventionally 60, flattens the difference between the very top positions so that a document appearing respectably in both lists can outrank one that is first in a single list.

def rrf(rankings, k=60):
    # rankings: one ordered list of doc ids per retriever
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            # only the position matters, so BM25 and cosine never meet
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Its virtues are that it needs no calibration, cannot be destabilised by one engine's score distribution shifting, and extends to three or more retrievers unchanged. Its cost is that it throws away real information: a lexical match that is overwhelmingly strong and one that barely cleared the threshold both occupy rank one. Where that matters, a weighted variant lets you favour one retriever, and the weight is a parameter you fit on a gold set rather than a constant you reason about. Weighting is also how you encode a query-type intuition without building a router — but if a query is a bare identifier, routing it to the lexical index alone will beat any fusion, so a cheap classifier on query shape is often worth more than tuning the fusion.

Two practical details decide whether this works. Retrieve deeper than you display, since fusion can only promote a document one of the retrievers actually returned, and a document ranked 40th lexically has to be in the list to be rescued. And deduplicate before fusing, because chunk overlap means the same passage appears twice under different chunk identifiers, and near-duplicates that both rank well will occupy your whole result set with one piece of content.

Prove it on the queries that were failing

The standard mistake is shipping hybrid retrieval as an obvious improvement and never measuring it, at which point nobody can say whether the added latency and operational surface bought anything. Evaluate three configurations on the same gold set — dense alone, lexical alone, fused — and segment the results by query type. The expected picture is that fusion is a modest average gain and a large gain on a specific slice, and if you cannot find the slice, you have paid for a second engine to produce the same answers.

The two retrievers are complements rather than competitors, so combine their rankings rather than their scores. Once you try to add a cosine similarity to a BM25 score you are inventing a normalisation that quietly changes with every query.