Loading...
Loading...
Browse 12 real-world technical and behavioral interview questions about Retrieval. Review scenarios, edge cases, and architectural best practices.
Source the questions from real users and from randomly sampled documents rather than from the team, label each with a quotable answer string anchored to a document rather than a chunk id so the set survives re-chunking, and hold part of it back so you are not tuning against your own measurement.
Because you cannot score a million-item catalogue per request inside a few tens of milliseconds. Retrieval reduces millions to hundreds with a cheap model judged on recall, then a far more expensive ranker orders those hundreds and is judged on the ordering it produces at the top of the list.
Almost always in retrieval rather than generation, and most often in chunking: if the answer was split across a boundary, or sits in a chunk whose surrounding context was stripped, no prompt can recover it. Evaluate the retrieval half separately, because it has a yes-or-no answer that needs no model to grade.
Walk three gates in order for that one request — was the supporting passage retrieved, did it survive into the rendered prompt, does the answer contradict it. Each gate blames a different stage, which is why the diagnosis depends on having logged both the retrieved chunk ids and the prompt as sent.
No. Cost and latency scale with what you send, accuracy over a long context is uneven rather than flat, and a corpus has no upper bound while a window does. Long context did change the job - it absorbs whole documents and forgives loose recall - but it replaces chunking, not retrieval.
Subtract the answer reservation and the fixed prompt first, give retrieval a token allowance rather than a passage count, and let conversation history have what remains under an explicit eviction policy - because history is the only term that grows every turn while everything else stays flat.
Chunk boundaries decide what can ever be retrieved, so you derive them from the questions the corpus must answer and from the document's own structure, then confirm the choice with retrieval recall on a gold set rather than by eye.
Read the failing queries first and sort them into causes, then apply the matching lever — rewriting for vague queries, filters for scope errors, smaller retrieval units for buried answers, lexical retrieval for literal terms, a reranker where recall sits below the cut — each with the metric that proves it worked.
Decide from the task. For grounded work over your own corpus, quality is set by what reaches the context, so retrieval dominates model size and a smaller model usually suffices. Keep the larger model for long multi-step reasoning, and treat routing as a second system to evaluate and monitor.
Ask what is missing. If the answer changes when your documents change, you need retrieval; if the model knows enough but will not behave, format or specialise the way you need, you fine-tune. Prompting comes first either way because it costs an afternoon and tells you which of the other two you actually need.
An embedding places text in a space where distance approximates the similarity its training data taught it. It does not encode relevance, and it represents exact identifiers, negation and numeric constraints so weakly that dense retrieval alone is unreliable wherever those decide the answer.
It earns it when your gold set shows recall at 50 is much better than recall at 5, because that gap is what reranking converts into answer quality. The cost is a model pass per candidate with nothing precomputable, so the latency budget divided by measured per-batch cost sets the candidate count.