Basic RAG often starts with one index and one similarity score. That is a reasonable prototype. It is not a universal retrieval architecture. Exact product codes, names, error strings, and legal phrases can be better served by lexical matching. Paraphrases and conceptually related wording can benefit from dense vectors. A reranker can spend more computation on a small candidate set than an initial index can afford across the whole corpus.
The goal is not to make retrieval complicated. The goal is to identify which failure a new stage fixes, measure that change on labelled queries, and keep the simplest architecture that meets the application’s needs.
Learning objectives
You will compare lexical and dense-like ranked lists, combine them with reciprocal-rank fusion, and understand why reranking is a second-stage operation. You will calculate recall at k, reciprocal-rank style measures, and precision on a tiny evaluation set. The local example is in content/build-with-ai-agents/advanced-retrieval.examples.
No external index or model is required. The dense run is a deterministic fixture so the mechanics of run comparison and fusion are visible without pretending a toy vector is a production embedding model.
Prerequisites
You should understand the RAG pipeline, embeddings, cosine similarity, and context budgets. Basic Python is enough for the example.
Lexical and dense signals
Lexical retrieval rewards shared terms. BM25-style methods account for term frequency, document frequency, and document length rather than treating every overlap as equal. They are strong for exact strings and often provide a reliable baseline.
Dense retrieval maps queries and passages into a vector space. It can match paraphrases that share few words, but it can also blur distinctions that matter: two products with similar descriptions, two policy versions, or a code with one character changed. Dense similarity should be evaluated alongside lexical cases, not only semantic paraphrases.
Hybrid search combines signals. The raw scores may not be comparable: a BM25 score and a cosine score have different ranges and meanings. Rank fusion avoids pretending their scales align. Reciprocal rank fusion gives each document credit based on its position in each ranked list. It is simple, robust, and still needs evaluation.
Toy problem / implementation
The example uses three small documents and a query about a duplicate charge. It computes a lexical run, accepts a deterministic dense-like run, and fuses the rankings:
def rrf(runs, k=60):
scores = {doc: 0.0 for run in runs for doc in run}
for run in runs:
for position, doc in enumerate(run, start=1):
scores[doc] += 1 / (k + position)
return sorted(scores, key=lambda doc: (-scores[doc], doc))The function never sees raw scores. That is its point. It combines evidence about ordering without requiring a calibration relationship between rankers. In a real system, retain the individual runs and the fused score so a reviewer can see why a document moved.
Run it yourself
From the repository root:
cd content/build-with-ai-agents/advanced-retrieval.examples
./verify.shThe verifier runs two tests and the demo. The lexical run ranks refund first for the exact query, the dense-like fixture supplies a second ordering, and fusion returns the union. These outputs demonstrate mechanics, not general retrieval superiority.
Reranking is a second stage
A cross-encoder reads the query and candidate passage together and produces a relevance score. It can model detailed interactions that a precomputed vector cannot, but it is too expensive to run over every document in a large corpus. The common architecture retrieves a wider candidate set cheaply, then reranks the top candidates with the expensive model.
Late-interaction methods sit between independent vectors and full cross-encoding. They preserve more token-level matching information while supporting efficient candidate search. The choice depends on corpus size, latency budget, update rate, hardware, and the kinds of distinctions the queries require.
Reranking does not fix missing candidates. If the relevant passage never enters the candidate set, a perfect second-stage model cannot recover it. Measure candidate recall before celebrating a reranker. If recall is low, improve query formulation, chunking, filters, indexing, or the first-stage ranker.
Query categories change the winner
One query set can contain several retrieval problems. A product-code query rewards exact matching and punctuation handling. A paraphrased policy question rewards semantic matching. A question asking for the newest rule needs freshness and document-version filtering. A long multi-part question may need decomposition or multiple retrieval calls. Report metrics by category so a hybrid system can be tuned to the failures that matter rather than to one blended average.
Metadata is another retrieval signal, but it should not be confused with relevance. A query filter such as product=payments can remove impossible candidates before ranking. A freshness preference can break ties among documents that answer the same question. An access filter is a hard boundary. Keep hard filters separate from soft ranking features so a high similarity score can never override authorization.
Reranking cost is often dominated by candidate count and sequence length. Measure the distribution, not only the mean: p50 and p95 latency, token counts, cache hit rate, and queue time. A reranker that improves MRR by a small amount but pushes tail latency beyond the product’s response budget may be a regression. Cache stable candidates carefully, with invalidation tied to source and model versions.
Do not train or tune a reranker on the same judgement set used for the final claim. Relevance labels are expensive and can encode annotator assumptions. Record who labelled a case, what “relevant” meant, and how disagreements were handled. For high-impact knowledge systems, review false negatives separately because a missed policy exception can matter more than many harmless false positives.
Evaluation metrics answer different questions
Recall at k asks whether at least one relevant item appears in the first k results. Mean reciprocal rank rewards the position of the first relevant result. Precision asks how much of the returned set is relevant. A downstream RAG system may care about recall because it needs the evidence somewhere in context, while a search UI may care about precision and first-result quality.
Create a labelled query set with relevant document ids, query categories, exact-match cases, paraphrases, long questions, permissions, and no-answer cases. Compare the baseline and candidate system on the same set. Report per-category results; an average can hide a catastrophic identifier failure behind easy semantic queries.
Use ablations: lexical only, dense only, fused, and fused plus reranker. Keep chunking and filters fixed while comparing rankers. Then test the full application, because a retrieval improvement can become a context-budget regression when the candidate set grows.
For a small corpus, a carefully implemented lexical baseline may be the best production choice. It is inspectable, cheap, and strong on exact terms. Start adding dense retrieval when labelled paraphrase failures justify it. Add fusion when the two rankers recover complementary relevant documents. Add a reranker when the candidate set is broad but the final ordering is consistently wrong. This staged rule keeps architecture proportional to evidence.
A retrieval release gate
A release gate can require minimum recall at the context size the generator can actually accept, no unauthorized results in a permission fixture, a maximum p95 ranking latency, and no regression on exact identifiers. Add a small set of must-retrieve queries for important policies and a no-answer set where the correct behavior is abstention. A single MRR number should not be allowed to hide a failure on those cases.
When comparing runs, keep the corpus snapshot and chunking version fixed. If the source changes, record whether a metric change came from the ranker or the data. Store run files in a simple format containing query id, document id, rank, score, and system version. This makes it possible to inspect a result manually, reproduce a fusion decision, and compare a new index without rebuilding the entire application.
The final retrieval list should remain explainable enough for operations. Preserve which stage introduced each candidate, whether the document passed filters, and whether a reranker changed its position. This metadata need not be shown to every end user, but it should be available to a reviewer investigating why the answer used one passage instead of another.
That evidence supports faster debugging and safer model changes.
It also keeps retrieval quality claims tied to inspectable, versioned, reproducible production artifacts consistently, always, explicitly.
Real-world application: enterprise retrieval
An enterprise knowledge system can route query types. Exact identifiers and quoted policy text can favor lexical search. Broad conceptual questions can include dense candidates. A hybrid stage can merge them, and a reranker can score the small union. Metadata filters for tenant, product, status, and document date should be applied before or during retrieval according to the index’s guarantees.
Do not let a more sophisticated ranker bypass permissions or freshness. Preserve the source metadata through every stage. Log the individual rank positions, fusion result, reranker version, selected chunks, and final context. If a user challenges an answer, the system should reconstruct which evidence won and why.
When not to add a reranker: if the corpus is small, lexical search is already above the task threshold, latency is strict, or no labelled failures identify a ranking problem. A reranker adds model cost, deployment complexity, and another versioned artifact. It should earn its place with evidence.
The same reasoning applies to query rewriting and decomposition. Rewriting can improve recall for a conversational question, but it can also remove an exact identifier or introduce an unsupported interpretation. Keep the original query, rewritten queries, and retrieved evidence in the trace. If a query is decomposed into multiple searches, join the results with provenance rather than flattening them into an unexplained list.
Failure modes and debugging
- Score-scale fusion. Do not add raw BM25 and cosine scores without calibration.
- Candidate recall ceiling. A reranker cannot recover a missing candidate.
- Metric mismatch. Choose recall, MRR, or precision based on the product failure.
- Easy evaluation set. Include identifiers, near duplicates, and no-answer queries.
- Permission after ranking. Apply authorization before context assembly and test it per tenant.
- Reranker overfits. Keep a held-out set and compare multiple query categories.
- Latency surprise. Measure candidate count, reranker time, and tail latency separately.
Limitations / when not to use
Advanced retrieval is unnecessary when a small, stable corpus and a simple search baseline meet the task requirement. It is also a poor substitute for improving source quality, chunk boundaries, metadata, or query wording. More stages create more ways to lose provenance and more components to monitor.
The local experiment does not run BM25, embeddings, or a cross-encoder. It demonstrates rank fusion and evaluation structure only. Use real labelled data and the exact production components before making a retrieval-quality claim.
Exercises
- Add a no-answer query and verify that every ranker can abstain.
- Implement a small BM25 scorer and compare it with overlap counts.
- Add a category-specific report for exact identifiers versus paraphrases.
- Measure how top-k changes context token count and answer support.
Next in this path
The next article is AI Agents: Models, Tools, State, and the Execution Loop. It uses retrieval and tools inside a bounded loop, then makes state and stopping rules explicit.
Sources
The research dossier is advanced-retrieval. Lexical ranking follows BM25. Late interaction follows Khattab and Zaharia, ColBERT. Cross-encoder behavior and retrieve-rerank architecture are documented by Sentence Transformers and retrieve-and-rerank. Evaluation practice follows TREC, the Stanford IR book, and ranx. Rank fusion follows Reciprocal Rank Fusion.
Research and code were last verified on 2026-08-05. The local ranking runs are illustrative and not a production benchmark.