hardDesignScenarioSeniorStaffLead
Latency on a checkout endpoint has tripled and nobody knows why. What should your metrics, logs and traces already have told you before anyone opens an editor?
6 min readUpdated 2026-08-15
Answer
Short answer
Metrics tell you something is wrong and when it started, traces tell you which hop owns the latency, logs tell you what happened to one request. Cardinality is what makes metrics expensive, sampling is what makes traces affordable, and the goal is answering new questions from data already held.
Keep observability explicit in the answer because that is the concept the interviewer is actually trying to test.
Was this answer helpful?
Likely follow-ups
- Your p99 is fine and a specific customer says the product is unusable. How do you find their requests?
- How would you keep a per-customer latency view without putting the customer id in a metric label?
- Where do you put the sampling decision when a trace crosses a team boundary and the other team samples at a different rate?
- What would you delete first if the observability bill were larger than the production bill?
Related questions
- Engineers added a user_id label to a few metrics and the observability bill went up tenfold. Explain what actually happened, and how you would fix it without losing the ability to debug.hardAlso on observability and metrics5 min
- What do you instrument and alert on for an LLM feature running in production?mediumAlso on observability and tracing5 min
- Your observability bill is now bigger than your compute bill. What do you cut first, and what must you never lose?hardAlso on sampling5 min
- A document was updated an hour ago and the assistant is still quoting the old version. Walk me through the diagnosis.hardAlso on observability6 min