Top 3 Experiment tracking Interview Questions
Browse 3 real-world technical and behavioral interview questions about Experiment tracking. Review scenarios, edge cases, and architectural best practices.
Six months after a decision, a regulator asks how your model arrived at it. What must you have logged at the time?
The prediction record must name the model version, the exact feature vector as served with its values, the identifier of the code that computed those features, the decision and any thresholds or overrides applied. Reconstruction after the fact is not possible, because feature pipelines and reference data have moved on.
What has to be captured for a training result to be reproducible?
Five things: the data version, the code commit, the resolved configuration, the pinned environment and the seeds. Even with all five, GPU kernels are often nondeterministic, so what you can promise is a statistically equivalent rerun and a recoverable artefact, not a bitwise identical one.
Why isn't a notebook a record of an experiment?
A notebook records the cells you wrote, not the order you ran them or the state they ran against, so the saved file frequently cannot reproduce its own outputs. Keep notebooks for exploration but promote anything you intend to defend into a parameterised script that logs its inputs to a tracking store. It also connects experiment tracking to the point an interviewer is testing.