Top 12 Retrieval Interview Questions
Browse 12 real-world technical and behavioral interview questions about Retrieval. Review scenarios, edge cases, and architectural best practices.
How do you build a gold set for a retrieval system?
Source the questions from real users and from randomly sampled documents rather than from the team, label each with a quotable answer string anchored to a document rather than a chunk id so the set survives re-chunking, and hold part of it back so you are not tuning against your own measurement.
Why is a recommender built as candidate generation followed by ranking rather than as one model?
Because you cannot score a million-item catalogue per request inside a few tens of milliseconds. Retrieval reduces millions to hundreds with a cheap model judged on recall, then a far more expensive ranker orders those hundreds and is judged on the ordering it produces at the top of the list.
Your RAG system cannot answer a question whose answer is definitely in the documents. Where is the fault?
Almost always in retrieval rather than generation, and most often in chunking: if the answer was split across a boundary, or sits in a chunk whose surrounding context was stripped, no prompt can recover it. Evaluate the retrieval half separately, because it has a yes-or-no answer that needs no model to grade.
A user reports a wrong answer. How do you work out whether retrieval or generation failed?
Walk three gates in order for that one request — was the supporting passage retrieved, did it survive into the rendered prompt, does the answer contradict it. Each gate blames a different stage, which is why the diagnosis depends on having logged both the retrieved chunk ids and the prompt as sent.
Models take a million tokens of context now. Does that remove the need for retrieval?
No. Cost and latency scale with what you send, accuracy over a long context is uneven rather than flat, and a corpus has no upper bound while a window does. Long context did change the job - it absorbs whole documents and forgives loose recall - but it replaces chunking, not retrieval.
How would you budget the context window for a chat feature backed by retrieval?
Subtract the answer reservation and the fixed prompt first, give retrieval a token allowance rather than a passage count, and let conversation history have what remains under an explicit eviction policy - because history is the only term that grows every turn while everything else stays flat.
How do you choose a chunking strategy for a document corpus?
Chunk boundaries decide what can ever be retrieved, so you derive them from the questions the corpus must answer and from the document's own structure, then confirm the choice with retrieval recall on a gold set rather than by eye.
Retrieval keeps returning plausible but unhelpful passages. How do you improve it without changing the model?
Read the failing queries first and sort them into causes, then apply the matching lever — rewriting for vague queries, filters for scope errors, smaller retrieval units for buried answers, lexical retrieval for literal terms, a reranker where recall sits below the cut — each with the metric that proves it worked.
Do you use one large model, or a smaller model with retrieval? How do you decide?
Decide from the task. For grounded work over your own corpus, quality is set by what reaches the context, so retrieval dominates model size and a smaller model usually suffices. Keep the larger model for long multi-step reasoning, and treat routing as a second system to evaluate and monitor.
How do you decide between prompting, retrieval and fine-tuning?
Ask what is missing. If the answer changes when your documents change, you need retrieval; if the model knows enough but will not behave, format or specialise the way you need, you fine-tune. Prompting comes first either way because it costs an afternoon and tells you which of the other two you actually need.
What does a text embedding encode, and what does it fail to encode?
An embedding places text in a space where distance approximates the similarity its training data taught it. It does not encode relevance, and it represents exact identifiers, negation and numeric constraints so weakly that dense retrieval alone is unreliable wherever those decide the answer.
Where does a cross-encoder reranker earn the latency it costs you?
It earns it when your gold set shows recall at 50 is much better than recall at 5, because that gap is what reranking converts into answer quality. The cost is a model pass per candidate with nothing precomputable, so the latency budget divided by measured per-batch cost sets the candidate count.