A vector answers one question: what is similar to this. Everything else a retrieval system has to do — restrict to documents this user may see, prefer the current version of a policy, cite a page, drop chunks from a deleted file, explain why a passage was returned — is served from the fields stored beside the vector. Treating metadata as bookkeeping is what produces an index that retrieves well in a demonstration and cannot be shipped.
The way to argue this in an interview is not to recite a schema but to attach each field to something it enables. If you cannot say which query or which guarantee needs a field, you do not need the field.
The fields, and what each one buys
Source document identity and version. Not a filename but a stable identifier from the system of record, plus a version or content hash. The identifier lets you delete or replace every chunk of a document in one operation, which is the mechanic behind incremental updates. The hash lets you skip re-embedding a document whose bytes have not changed, which is what makes a nightly re-crawl affordable.
Section path. The heading chain the author wrote: document title, then heading, then sub-heading. It has two uses. Prefixed into the embedded text it fixes fragments that would otherwise match nothing, because a table row or a bare sentence inherits its subject. Stored as a field it lets a reader see where a passage sits, which is most of what makes a citation trustworthy.
A stable anchor. The thing that turns a citation into a link. A page number for a PDF, a heading identifier for HTML, a sheet name and cell range for a spreadsheet, a slide index for a deck. It has to survive re-ingestion of the same document, which is why a character offset into an extracted string is a poor anchor and a page or heading is a good one.
Effective date and ingestion date, kept separately. These are different facts and conflating them is a real defect. The effective date is when the content became true — the date a policy takes effect, a contract's commencement. The ingestion date is when you indexed it. A question about last quarter's rates needs the first; a question about whether the index is stale needs the second. One field cannot answer both.
The permission set that governed the original. Whatever the source system used to decide who may read the document: group identifiers, an access-control list, a classification label, a tenant identifier. This is the field people leave until later, and leaving it until later means the only available enforcement is filtering results after retrieval, which changes how many results a user gets and leaks the existence of documents through result counts and scores.
Content type and language. Type — prose, table row, code, slide notes — because different chunk types deserve different treatment at rerank time and different rendering in the answer. Language because a multilingual corpus needs it both for filtering and for choosing how to process a query.
The chunk identifier is worth a second look: because it derives from the document, its version and its position, deleting a document version is a prefix operation rather than a search.
Why the reindex threat is the whole argument
The reason to over-capture at ingestion rather than to add fields when a requirement appears is asymmetric cost. Adding a field that can be derived from something you already stored is a cheap backfill. Adding a field that only existed while the original file was open is not: the page number, the header row above a value, the owner recorded in the source system, the permission set at the time. Recovering any of those means fetching every source document again, parsing it again, and rebuilding the index — and if you must re-parse, you are re-chunking, and if you are re-chunking, you are re-embedding, which is the most expensive operation in the pipeline.
That is what makes this a design decision rather than a detail. The question to ask during ingestion is not "what do we need" but "what will be impossible to obtain in six months", and the answer is anything that lives in the source system rather than in the extracted text. Storing a field you never query costs a few bytes per chunk. Omitting one costs a full pipeline run over the entire corpus.
Note also that not every field belongs in the vector store. Volatile facts — a permission grant that changes weekly, a document's current owner — are better held by reference, with the vector store keeping a stable key and the authoritative value resolved at query time. A copied permission is a permission that goes stale silently, and silently stale access control is the worst failure mode in this list.
Capture at ingestion everything that only exists while the original document is open. Fields you can derive later are cheap to add; fields that lived in the source system are a full re-parse, re-chunk and re-embed away.
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).