hardScenarioMidSeniorStaff
Consumer lag on your orders topic climbs every morning and never clears before the next peak. Work through it.
6 min readUpdated 2026-08-15
Answer
Short answer
Lag is the integral of arrival rate minus service rate, so establish which of the two moved and whether the consumer is slow or stalled. Check lag per partition before per group, rule out a rebalance loop, and remember that adding consumers beyond the partition count does nothing.
Kafka lag is a symptom, not the diagnosis. Good Kafka analysis separates slow processing, partition skew, downstream backpressure, rebalance churn, and insufficient parallelism. Weak Kafka answers add consumers before checking whether the topic has enough partitions or whether one hot key owns the lag.
Was this answer helpful?
Likely follow-ups
- Lag is concentrated in one partition out of twenty-four. What are your options?
- You need to catch up on four hours of backlog today. What do you do differently from the steady-state fix?
- One message cannot be processed and never will be. Where does it go, and who finds out?
- How would this diagnosis differ on SQS rather than Kafka?
Related questions
- Your broker stops accepting writes for twenty minutes. What does each producer do, and what should you have decided before it happened?hardAlso on kafka and backpressure6 min
- A transform has been writing wrong revenue figures for three days and six downstream tables have consumed it. How do you backfill the corrected data without double-counting anything?hardAlso on partitioning4 min
- Your services talk through events and one consumer has been down for an hour. What has it missed, and how does it catch up?hardAlso on kafka6 min
- A dependency that normally answers in 80ms starts taking eight seconds. What in your service reacts, and in what order?hardAlso on backpressure7 min