hardScenarioDesignSeniorStaff
How do you architect a strictly low-latency, real-time pipeline to ingest, embed, and index tens of thousands of unstructured documents per second into a vector database?
3 min readUpdated 2026-08-15
Answer
Short answer
Designing a robust streaming architecture for embedding generation, detailing GPU adaptive batching, Kafka-driven backpressure management, and asynchronous HNSW indexing.
Was this answer helpful?
Likely follow-ups
- How would you handle a mid-stream change to the embedding model itself, given that old and new vectors no longer share a coordinate space?
- A burst of documents arrives that exceed the embedding model's context window. What breaks, and how do you catch it before it corrupts the index?
- How do you choose the chunk overlap size, and what degrades in retrieval quality if you get it wrong in either direction?
Related questions
- How do you achieve true exactly-once semantics in Flink across source, state, and sink without cratering throughput?hardAlso on data-engineering and streaming2 min
- Rate-limit an LLM gateway where one request can cost a hundred times another.hardAlso on streaming5 min
- How would you design a data lakehouse to handle petabytes of data with frequent GDPR right-to-be-forgotten requests and rapid schema evolution without sacrificing query latency?hardAlso on data-engineering3 min
- How do you implement dynamic PII masking across a highly decentralised data mesh without destroying the analytical utility of the data for downstream machine learning workloads?hardAlso on data-engineering2 min