Top 4 Reproducibility Interview Questions
Browse 4 real-world technical and behavioral interview questions about Reproducibility. Review scenarios, edge cases, and architectural best practices.
You are dropped into an unfamiliar repository with one failing business rule and an AI assistant that only gives hints. What do you do in the first ten minutes?
Debugging an unfamiliar codebase starts by running the failing test, then reading it as the rule and tracing its call path to the code that computes the wrong value. Let the size of the error narrow the cause, ask the assistant to locate and explain rather than patch, and prove the fix with a test that failed before.
Six months after a decision, a regulator asks how your model arrived at it. What must you have logged at the time?
The prediction record must name the model version, the exact feature vector as served with its values, the identifier of the code that computed those features, the decision and any thresholds or overrides applied. Reconstruction after the fact is not possible, because feature pipelines and reference data have moved on.
What has to be captured for a training result to be reproducible?
Five things: the data version, the code commit, the resolved configuration, the pinned environment and the seeds. Even with all five, GPU kernels are often nondeterministic, so what you can promise is a statistically equivalent rerun and a recoverable artefact, not a bitwise identical one.
Why isn't a notebook a record of an experiment?
A notebook records the cells you wrote, not the order you ran them or the state they ran against, so the saved file frequently cannot reproduce its own outputs. Keep notebooks for exploration but promote anything you intend to defend into a parameterised script that logs its inputs to a tracking store. It also connects experiment tracking to the point an interviewer is testing.