How do you keep a retrieval index current as the underlying documents change?

Change detection decides everything downstream

The pipeline you can afford is determined by how you learn that something changed. A push signal — a change feed from the database, a webhook from the content system, a log-based capture stream — tells you which documents moved and lets the rest of the corpus sit untouched. A pull signal means crawling an inventory and comparing it against what you already hold, which is tractable if the source will give you a cheap listing with versions or entity tags, and miserable if it will only give you the bytes.

Whichever direction the signal arrives from, you need a manifest: one row per source document recording its source identifier, the version or entity tag you last saw, a hash of the extracted text, the chunk identifiers you produced from it, the embedding model and version used, and the timestamp at which it became searchable. Nearly every freshness question later becomes a query against that table. Note that the hash should cover the extracted text, not the raw file, because re-exports, re-scans and format migrations change bytes constantly without changing a word of meaning, and a byte hash will have you re-embedding a quarter of the corpus every time someone runs a bulk conversion.

Modification timestamps deserve specific suspicion. They are frequently rewritten by the storage layer, by sync clients, and by migrations, and they are frequently not updated when content changes through an API rather than a file write. Use them to narrow a candidate set, never as proof that content is unchanged.

Upsert at document granularity, diff at chunk granularity

The awkward property of an edited document is that chunk boundaries move. Insert a paragraph on page one and every subsequent chunk shifts, so a positional chunk identifier such as doc-42-chunk-7 now points at different text than it did yesterday. Treating that as an in-place update leaves the tail of the document holding a mix of old and new content, and the failure is silent because every row still looks valid.

The reliable shape is: re-parse and re-chunk the whole document, then reconcile the resulting set against the manifest as a set operation — retire chunk rows that are no longer produced, insert the ones that are new, and leave alone the ones whose content hash is unchanged. Deriving the chunk identifier from the document identifier plus a hash of the chunk's own text makes that reconciliation trivial and means a one-line edit near the end of a long document re-embeds two chunks rather than sixty. Embedding is the expensive step, so the diff is where the money is. Do the retire-and-insert as one logical operation, because the window in which a document has lost its old chunks and not yet gained its new ones is a window in which it answers nothing.

flowchart LR
  A[Source change event] --> B[Parse and chunk]
  B --> C{Chunk hash in manifest}
  C -->|unchanged| D[Skip embedding]
  C -->|new or changed| E[Embed and upsert]
  A --> F{Delete or unshare}
  F -->|yes| G[Write tombstone]
  G --> H[Query filter excludes]

The edge worth studying is the one from tombstone straight to the query filter, bypassing the embed path entirely. Removal has to take effect at read time before any index compaction happens, because compaction is asynchronous and you do not control when it finishes.

A deleted document that still answers questions is an incident

Every other kind of staleness is a quality problem you can apologise for. This one is not. If a document has been deleted, unshared, retracted or superseded and your index still serves its text as grounding, the assistant will quote it confidently and cite it, and the fact that the source of truth no longer contains it is exactly the point. A retracted price, a rescinded policy, a record that was deleted on legal instruction, a file whose sharing was revoked — none of those are cache misses. Say this out loud in the interview, because it is the sentence that separates someone who has run a retrieval system from someone who has built one.

Practically that means three things. Deletions jump the queue: they are processed ahead of routine reindexing rather than behind a backlog of updates. Deletion is recorded as a tombstone in the manifest and enforced as a predicate on retrieval, so the removal is effective the moment it is recorded rather than whenever the vector store finishes rewriting segments. And because a tombstone in the manifest also stops a later crawl from cheerfully re-ingesting a file it still sees in a stale listing, resurrection becomes a bug you can detect rather than one you discover in an answer.

Many sources will never tell you about a deletion at all; the file simply stops appearing. That is why a periodic reconciliation sweep — diff the manifest's document set against a current source inventory, tombstone the orphans — is not optional housekeeping. It is the only mechanism that catches the class of deletion your change feed structurally cannot report.

Lag is a published number or it is an accident

"Near real time" is not an answer. Pick a target — for example, edits searchable within five minutes at the ninety-fifth percentile, deletions within thirty seconds — and then build the measurement that tells you whether you are meeting it. The cheapest instrument is a canary document that a scheduled job edits and then polls for through the ordinary retrieval path, which gives you end-to-end lag including every queue and cache rather than the stage timings you would otherwise report to yourself. Carry source_updated_at and indexed_at on every chunk so a retrieved passage can be dated, and so the assistant can qualify an answer whose evidence is older than the user's question implies.

Different classes of change deserve different targets, and saying so is a sign of judgement. Permission changes and deletions are correctness. Routine content edits are quality. A reasonable design lets the first two pre-empt the third when a bulk operation floods the pipeline, and sheds or defers the third rather than letting a forty-thousand-document migration delay a single revocation.

The freshness question an auditor asks is never "how fast are your updates" — it is "how long can something you deleted keep answering", and you should already know that number.