Skip to content
Preptima

Experiment Tracking & Reproducibility interview questions

What has to be captured for a result to be reproducible, data and code versioning, seeds and nondeterminism, and why a notebook is not a record.

3 questions in MLOps & ML Platform.

medium2

mediumConceptDesign

What has to be captured for a training result to be reproducible?

Five things: the data version, the code commit, the resolved configuration, the pinned environment and the seeds. Even with all five, GPU kernels are often nondeterministic, so what you can promise is a statistically equivalent rerun and a recoverable artefact, not a bitwise identical one.

4 minmid, senior, staff
mediumConceptScenario

Why isn't a notebook a record of an experiment?

A notebook records the cells you wrote, not the order you ran them or the state they ran against, so the saved file frequently cannot reproduce its own outputs. Keep notebooks for exploration but promote anything you intend to defend into a parameterised script that logs its inputs to a tracking store.

4 minentry, mid, senior

hard1