MLOps interview questions

Experiment tracking, model registries, deployment patterns, drift detection, and retraining triggers.

8 questions in Data & AI Engineering.

Data and AIA 22-minute read on how the topic fits together, before the questions.Data and AI engineering fundamentals58 short answers on one page, for revising rather than studying.

hard8

hardDesignScenario

The recommender is good for users with history and useless for the ones who just signed up. What do you serve them?

Serve context instead of history: popularity conditioned on the country, device and referrer you already have, then re-rank on the first two clicks of the session with a model that needs no user identity. Reserve slots for exploration, and segment every offline metric by history length, because a metric averaged over users with history cannot see this cohort at all.

7 minmid, senior, staff

hardScenarioDesign

How do you architect a serving system for a 70B parameter LLM to maximize GPU throughput without violating strict time-to-first-token (TTFT) latency SLAs?

An authoritative deep dive into LLM inference optimization, exploring continuous batching, paged attention for KV cache memory management, and tensor parallelism scheduling.

3 minsenior, staff

hardConceptScenario

A model has been degrading for months. How do you decide it should be retired rather than retrained again?

Retraining only helps when the model's premise still holds. If the proxy it predicts no longer stands for the outcome you care about, or the feature it depended on has stopped being generated, more data cannot recover it, and the decision is retirement plus a stated fallback. Use this model lifecycle answer to show the decision, trade-off, and evidence rather than a memorised definition.

5 minsenior, staff, lead

hardScenarioDesign

Would you deploy LoRA adapters or a Mixture of Experts (MoE) architecture to serve hundreds of highly specialised enterprise domains from a single foundation model, and why?

An analysis of the cost, performance, and complexity trade-offs between LoRA fine-tuning and MoE architectures for multi-tenant domain adaptation. Use this cost optimization answer to show the decision, trade-off, and evidence rather than a memorised definition. It also connects finetuning to the point an interviewer is testing.

3 minsenior, staff

hardScenarioDesign

How would you architect a Kubernetes cluster to manage a heterogeneous fleet of GPUs simultaneously running massive distributed training jobs and latency-sensitive inference services without catastrophic degradation?

An in-depth look at designing a robust Kubernetes orchestration layer to optimise GPU utilisation across diverse, conflicting machine learning workloads.

2 minsenior, staff

hardScenarioDesign

How would you architect a defence-in-depth strategy to prevent model inversion and training data extraction attacks on a publicly accessible generative AI service fine-tuned on highly sensitive corporate data?

Exploring robust security architectures to prevent attackers from extracting sensitive training data from generative AI models.

2 minsenior, staff

hardScenarioDesign

A model scored well in validation and performs badly in production, with no errors anywhere. What do you check first?

Check whether training and serving compute the same features from the same definitions, because the usual cause is two implementations that disagree slightly. Nothing raises an error when they diverge - the model receives plausible numbers that mean something different from the ones it learned on.

5 minmid, senior, staff

hardDesignScenario

What has to be in place before you would let a model serve live traffic?

Features must be computable at decision time, training and serving must share one transformation implementation, the model and dataset version must be pinned together, drift and realised performance monitored separately because labels arrive late, and rollback must move model and features as one unit.

6 minmid, senior, staff, lead