AI Engineer Roadmap All lessons Interactive Quiz GitHub

Ensemble Retrieval

Intermediate · 7 min read · Updated · By Saurabh Pathak

In short: Ensemble retrieval runs several different retrievers (usually keyword and vector) on the same question and fuses their results into one ranked list.

Play the interactive lesson: Ensemble retrievalRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. When to use it / when not to
  6. Common mistakes
  7. Recap
  8. Interview questions
  9. Go deeper

Level: Intermediate · Reading time: 7 min · Phase: 2 — Advanced RAG

Before you start: Hybrid Search, BM25 and Bi-Encoder. Ensemble retrieval generalises hybrid search to any set of retrievers.

Explain it like I'm new

When you hire, you might ask two interviewers: one checks technical skills, one checks communication. Each misses things the other catches. Combining their rankings gives a better shortlist than either alone.

Retrievers have different blind spots:

Ensemble retrieval, often called hybrid search when it combines exactly these two, runs both and merges the lists. Many vector databases offer hybrid search built in.

Words to know:

Watch it

Ensemble retrieval: Several retrievers run together and their rankings are fused.
Ensemble retrieval: Several retrievers run together and their rankings are fused.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

  1. Send the query to each retriever (in parallel).
  2. Get a ranked list from each.
  3. Fuse:
    • Reciprocal Rank Fusion (RRF): score(d) = Σ w_i / (60 + rank_i(d)). Only ranks matter.
    • Weighted score fusion: normalise each retriever's scores to 0–1, then α·dense + (1−α)·sparse.
  4. Optionally rerank the fused top 50 down to 5.
flowchart LR
    Q[Query] --> B[BM25] --> F[Fusion<br/>RRF / weighted]
    Q --> V[Vector search] --> F
    F --> X[Optional reranker] --> T[Top k]

Weighted fusion worked example with α = 0.5:

DocBM25 (normalised)Dense (normalised)Fused
A (contains "ERR_CONN_RESET")1.00.30.65
B (paraphrase of the problem)0.11.00.55
C0.40.40.40

Worked example

Question: "connection reset by proxy". Three documents: a mentions ERR_CONN_RESET and the proxy dropping connections, b says a connection keeps getting reset and to raise the proxy timeout, c is about TLS certificates. Real output from the code below:

  1. Keyword ranking: ['b', 'a', 'c']. b shares three query words (connection, reset, proxy). a and c share one each (proxy).
  2. Trigram ranking (overlapping 3-letter pieces, which are forgiving about spelling): ['a', 'b', 'c'].
  3. Fuse with RRF, weights 0.5 each, k=60:
    • b: 0.5/(60+1) + 0.5/(60+2)
    • a: 0.5/(60+2) + 0.5/(60+1), the same total.
  4. a and b tie because each ranked first once and second once. c is last in both lists. The final list is ['b', 'a', 'c'], with the tie broken by which document was seen first.

A document near the top of both lists beats one that is top of only one list. Ties like this are normal with two short lists.

Code example

A stdlib ensemble that fuses a keyword ranker and a character-trigram "semantic-ish" ranker with weighted RRF:

import re

DOCS = {
    "a": "ERR_CONN_RESET appears when the proxy drops idle connections.",
    "b": "If the connection keeps getting reset, raise the proxy idle timeout.",
    "c": "Configure TLS certificates for the proxy.",
}

def kw_rank(q):
    qw = set(re.findall(r"\w+", q.lower()))
    s = {d: len(qw & set(re.findall(r"\w+", t.lower()))) for d, t in DOCS.items()}
    return [d for d in sorted(s, key=lambda d: -s[d]) if s[d] > 0]

def trigram_rank(q):
    grams = lambda t: {t[i:i + 3] for i in range(len(t) - 2)}
    qg = grams(q.lower())
    s = {d: len(qg & grams(t.lower())) / len(qg | grams(t.lower())) for d, t in DOCS.items()}
    return sorted(s, key=lambda d: -s[d])

def weighted_rrf(rankings, weights, k=60):
    scores = {}
    for ranking, w in zip(rankings, weights):
        for r, d in enumerate(ranking, 1):
            scores[d] = scores.get(d, 0) + w / (k + r)
    return sorted(scores, key=lambda d: -scores[d])

q = "connection reset by proxy"
print(weighted_rrf([kw_rank(q), trigram_rank(q)], weights=[0.5, 0.5]))

See rrf_merge in adv_query_transform.py for the tested, unweighted version.

In real projects LangChain has an EnsembleRetriever that takes a list of retrievers and weights and fuses with weighted RRF; it is often paired with a BM25 retriever. Vector databases such as Weaviate, Qdrant, Elasticsearch/OpenSearch and others offer native hybrid search. See LangChain docs for current imports.

When to use it / when not to

FusionNeeds score normalisationTuning
RRFNoLittle (weights optional)
Weighted scoreYesα must be tuned on an eval set

Common mistakes

Recap

If you remember only three things:

Interview questions

1. Why combine BM25 with dense retrieval? <details><summary>Answer</summary> They fail differently. BM25 nails exact tokens (IDs, rare names) but misses paraphrases; dense retrieval handles meaning but can miss exact identifiers. Fusing them improves recall across query types. </details>

2. RRF or weighted score fusion? <details><summary>Answer</summary> RRF is robust and needs no normalisation, so it's a good default. Weighted fusion can do better when tuned on an eval set, but depends on careful score normalisation. </details>

3. Where does a reranker fit in a hybrid pipeline? <details><summary>Answer</summary> After fusion: hybrid retrieval maximises recall over a candidate set (say 50), then a cross-encoder reranks for precision before the top few go to the LLM. </details>

Go deeper

Prev: Parent Document Retriever · Next: Knowledge Graph RAG · Back to phase: Phase 2