mediumConceptDesignMidSeniorStaff
How do you keep a retrieval index current as the underlying documents change?
5 min readUpdated 2026-08-15
Answer
Short answer
Detect change at the source rather than by re-crawling everything, upsert at document granularity so moved chunk boundaries do not orphan rows, propagate deletions as explicit tombstones the query filter respects, and publish the source-to-searchable lag as a measured SLA.
Was this answer helpful?
Likely follow-ups
- Your source system emits no change events at all. How do you detect edits without re-embedding the corpus nightly?
- A single bulk edit touches forty thousand documents at once. What protects the pipeline and the query path?
- How do you make an answer state the age of the evidence it used?
- When is a full rebuild genuinely cheaper than incremental maintenance?
Related questions
- A document was updated an hour ago and the assistant is still quoting the old version. Walk me through the diagnosis.hardAlso on freshness and incremental-indexing6 min
- How do you build a gold set for a retrieval system?mediumAlso on rag5 min
- How do you choose a chunking strategy for a document corpus?mediumAlso on rag4 min
- What metadata do you store alongside each chunk, and what is each field for?mediumAlso on rag4 min