Loading...
Loading...
Browse 6 real-world technical and behavioral interview questions about Monitoring. Review scenarios, edge cases, and architectural best practices.
It depends entirely on where it stopped, because silence is normal on a long line haul and alarming between two hand-offs that should be minutes apart. The design that answers it is an expected next event with a deadline per milestone, rather than a threshold on time since the last scan.
Data drift is a change in the distribution of the inputs; concept drift is a change in the relationship between inputs and the outcome. You can detect data drift from inputs alone, concept drift needs labels or a performance signal, and only concept drift necessarily invalidates what the model learned.
Retrain when there is evidence the learned relationship no longer holds and you have data that reflects the new one. A fixed schedule is a fallback for when you cannot measure that, not a decision, and a drift alert is as likely to mean an upstream pipeline broke as it is to mean the world moved.
Monitor what is available immediately: input distributions and data quality, the prediction distribution, and proxy outcomes that correlate with the label but arrive sooner. Do all of it per segment, because a small segment failing badly is invisible in an aggregate.
Start by instrumenting outcomes, because you almost certainly cannot say what happened to any past alert. Delivered recall is model recall multiplied by the rate at which alerts are acted on, so at a fifteen per cent action rate a better model changes nothing that anyone experiences.
Check whether training and serving compute the same features from the same definitions, because the usual cause is two implementations that disagree slightly. Nothing raises an error when they diverge - the model receives plausible numbers that mean something different from the ones it learned on.