hardScenarioDesignSeniorStaff
How do you design a petabyte-scale telemetry pipeline that decouples generation from storage while aggressively curbing backend SaaS costs?
2 min readUpdated 2026-08-15
Answer
Short answer
An architectural blueprint for unified observability pipelines, focusing on edge processing, intelligent tail-based sampling, and vendor-agnostic telemetry.
Was this answer helpful?
Likely follow-ups
- How do you decide what to sample at the edge collector versus the gateway when the two tiers disagree on a trace's importance?
- What happens to this pipeline when a single misbehaving service starts emitting fifty times its normal log volume?
- How do you verify that tail-based sampling isn't silently discarding traces that matter to a specific customer?
Related questions
- Engineers added a user_id label to a few metrics and the observability bill went up tenfold. Explain what actually happened, and how you would fix it without losing the ability to debug.hardAlso on observability5 min
- A document was updated an hour ago and the assistant is still quoting the old version. Walk me through the diagnosis.hardAlso on observability6 min
- A regulator says this customer's data may not leave the country. How do you design for that?hardAlso on telemetry7 min
- How do you handle personal data in prompts, logs and traces for an LLM feature?mediumAlso on observability5 min