RAG, Retrieval & Agents

What this area actually covers

Retrieval-augmented generation is the practice of answering a question by first finding relevant material and then asking a language model to answer using only that material. Written out like that it sounds like a prompt technique. It is not: it is a search system, an ingestion pipeline, an index with an update problem, a permission model, and an evaluation problem with several stages, and the language model is the last and least interesting component in the chain.

This section covers that whole chain. How documents get parsed and split. What an embedding encodes and what it does not. How an approximate nearest-neighbour index trades recall for latency. Why dense vectors alone fail on exact terms and what to combine them with. How to tell whether a bad answer was a retrieval failure or a generation failure. How to keep an index honest when the underlying documents change and when they are deleted. How to enforce, inside retrieval, that a user only sees what they are entitled to see. And finally agents — systems that call tools in a loop rather than answering once — which belong here because tool use is the generalisation of retrieval and inherits all of its problems plus several of its own.

The reason this is a separate section from Generative AI & Large Language Models is a practical one. In a working enterprise AI system, the prompting and model-selection work is perhaps a fifth of the effort and the retrieval pipeline is most of the rest, and the two are examined differently: a GenAI interview asks how you evaluated the output, and a retrieval interview asks what your recall at ten was before reranking and after. The two sections overlap on evaluation and on injection because a retrieval system is a delivery mechanism for injected content, and each treats those from its own end.

There is an older, shorter treatment of this material on the site. The Data & AI Engineering section has an LLMs & RAG subsection from an earlier wave, which still stands and is a compressed introduction to what is now two sections. It has not been removed. If you read both, treat this as the detailed version.

Two things get wrongly folded in. The first is vector databases as a topic in themselves: choosing a vector store is a genuine decision with genuine trade-offs, and it is a much smaller decision than how you chunk and how you rerank, and candidates who prepare a comparison of vector stores have prepared for the least consequential question. The second is agents as a synonym for AI: an agent is one architecture, appropriate when a task genuinely requires several dependent steps whose sequence is not known in advance, and inappropriate — slower, more expensive, harder to debug, and less reliable — for tasks a single well-retrieved call handles.