Top 11 Rag Interview Questions
Browse 11 real-world technical and behavioral interview questions about Rag. Review scenarios, edge cases, and architectural best practices.
Your RAG assistant cites the wrong document, and sounds completely sure of itself. How do you find which stage failed?
Replay a wrong citation stage by stage: does the pointer resolve to the passage the model saw, does that passage support the claim, did the right document survive reranking, and did retrieval find it at all. A golden set with gold document ids and supporting spans turns each check into a per-stage metric.
You are building a support assistant over a knowledge base that changes every week. RAG or fine-tuning, and how do you decide?
Use RAG for the knowledge, because articles change weekly and retrieval lets you edit, withdraw and cite them without retraining. Fine-tune only for behaviour retrieval cannot fix, such as tone, escalation judgement or reply format, and decide by auditing real failures and evaluating each fix on its own slice.
How do you build a gold set for a retrieval system?
Source the questions from real users and from randomly sampled documents rather than from the team, label each with a quotable answer string anchored to a document rather than a chunk id so the set survives re-chunking, and hold part of it back so you are not tuning against your own measurement.
Your RAG system cannot answer a question whose answer is definitely in the documents. Where is the fault?
Almost always in retrieval rather than generation, and most often in chunking: if the answer was split across a boundary, or sits in a chunk whose surrounding context was stripped, no prompt can recover it. Evaluate the retrieval half separately, because it has a yes-or-no answer that needs no model to grade.
A better embedding model has come out. What does moving to it cost you?
Vectors from two models live in unrelated spaces, so there is no incremental path — you re-embed and reindex the whole corpus. The real cost is running both indexes side by side long enough to compare them on a gold set, keeping query and document models in step, and retuning thresholds calibrated on the old scores.
Your RAG system returns confident but wrong answers. How do you debug it?
Debug a RAG system with confident wrong answers by separating retrieval failure from generation failure. Check whether the gold passage was retrieved, then measure recall@k, reranking quality, faithfulness and abstention on a labelled eval set. It also connects retrieval evaluation to the point an interviewer is testing.
How do you choose a chunking strategy for a document corpus?
Chunk boundaries decide what can ever be retrieved, so you derive them from the questions the corpus must answer and from the document's own structure, then confirm the choice with retrieval recall on a gold set rather than by eye.
Your corpus is scanned PDFs, spreadsheets and slide decks. How do you ingest it?
Route each format to a parser that respects its structure, treat OCR output as text with a confidence score rather than as truth, convert tables into rows that carry their headers, and record a section path and stable anchor per chunk so every retrieved passage can be cited back to a place in the original.
How do you keep a retrieval index current as the underlying documents change?
Detect change at the source rather than by re-crawling everything, upsert at document granularity so moved chunk boundaries do not orphan rows, propagate deletions as explicit tombstones the query filter respects, and publish the source-to-searchable lag as a measured SLA.
What does a text embedding encode, and what does it fail to encode?
An embedding places text in a space where distance approximates the similarity its training data taught it. It does not encode relevance, and it represents exact identifiers, negation and numeric constraints so weakly that dense retrieval alone is unreliable wherever those decide the answer.
What metadata do you store alongside each chunk, and what is each field for?
Each chunk needs its source document and version, its section path, an anchor precise enough to open, effective and ingestion dates, and the permission set that governed the original — because filtering, citation, freshness and access control are all served from metadata rather than from the vector.