ML Fundamentals interview questions

Bias and variance, evaluation-metric selection, leakage, imbalanced data, and validation strategy.

4 questions in Data & AI Engineering.

Data and AIA 22-minute read on how the topic fits together, before the questions.Data and AI engineering fundamentals58 short answers on one page, for revising rather than studying.

medium3

mediumConceptScenarioCase Study

A churn model scores 0.94 AUC offline and barely beats the baseline in production. One of its strongest features is populated by a process that runs after the customer has already churned. Explain what happened and how you would fix it.

The feature encodes the outcome, so the model reads the answer rather than predicting it. Offline the label and the feature come from the same snapshot, which is why validation looks excellent; in production the feature is empty at scoring time and the signal vanishes. Fix by defining features by when they were available and validating on a forward time split.

5 minmid, senior, staff

mediumConceptScenario

Your fraud model is 99.4% accurate. Is that good?

Almost certainly not: if fraud is 0.6% of transactions, a model predicting 'not fraud' for everything scores 99.4% and catches nothing. The metric has to follow from which error costs more, which makes it a business decision expressed in numbers rather than a modelling one.

4 minmid, senior

mediumConceptScenario

How do you tell that a model is overfitting, and what do you do about it?

Detect overfitting by comparing training and validation performance on an honest split. Fix it with more or better data, simpler models, regularisation, early stopping and a final test set that is read only once. It also connects model validation to the point an interviewer is testing.

4 minentry, mid, senior

hard1

hardConcept

A more complex model beats your simple one by a small margin offline. How do you decide whether it is worth shipping?

Convert the offline gain into the decision it changes and the value of that change, then set it against what the model commits the team to for its whole life. A small metric improvement that no threshold or downstream action responds to is worth nothing, however real it is.

5 minmid, senior, staff