Top 6 Monitoring Interview Questions
Browse 6 real-world technical and behavioral interview questions about Monitoring. Review scenarios, edge cases, and architectural best practices.
A shipment has produced no scan events for eighteen hours. Is that a problem?
It depends entirely on where it stopped, because silence is normal on a long line haul and alarming between two hand-offs that should be minutes apart. The design that answers it is an expected next event with a deadline per milestone, rather than a threshold on time since the last scan.
What is the difference between data drift and concept drift, and does it change what you do?
Data drift is a change in the distribution of the inputs; concept drift is a change in the relationship between inputs and the outcome. You can detect data drift from inputs alone, concept drift needs labels or a performance signal, and only concept drift necessarily invalidates what the model learned.
How do you decide when to retrain a model?
Retrain when there is evidence the learned relationship no longer holds and you have data that reflects the new one. A fixed schedule is a fallback for when you cannot measure that, not a decision, and a drift alert is as likely to mean an upstream pipeline broke as it is to mean the world moved.
Your labels arrive months after the prediction, or never. What do you monitor?
Monitor what is available immediately: input distributions and data quality, the prediction distribution, and proxy outcomes that correlate with the label but arrive sooner. Do all of it per segment, because a small segment failing badly is invisible in an aggregate.
Your predictive maintenance system is live and the maintenance team has stopped acting on its alerts. How do you get that back?
Start by instrumenting outcomes, because you almost certainly cannot say what happened to any past alert. Delivered recall is model recall multiplied by the rate at which alerts are acted on, so at a fifteen per cent action rate a better model changes nothing that anyone experiences.
A model scored well in validation and performs badly in production, with no errors anywhere. What do you check first?
Check whether training and serving compute the same features from the same definitions, because the usual cause is two implementations that disagree slightly. Nothing raises an error when they diverge - the model receives plausible numbers that mean something different from the ones it learned on.