Loading...
Loading...
Browse 11 real-world technical and behavioral interview questions about Data leakage. Review scenarios, edge cases, and architectural best practices.
The feature encodes the outcome, so the model reads the answer rather than predicting it. Offline the label and the feature come from the same snapshot, which is why validation looks excellent; in production the feature is empty at scoring time and the signal vanishes. Fix by defining features by when they were available and validating on a forward time split.
Build the split to imitate the gap between training and deployment. Group all of an entity's rows onto one side when rows repeat per entity, split forward in time for anything temporal with an embargo covering the label window, and treat a plain random shuffle as the default that silently inflates almost every score.
Detect overfitting by comparing training and validation performance on an honest split. Fix it with more or better data, simpler models, regularisation, early stopping and a final test set that is read only once. It also connects model validation to the point an interviewer is testing.
One-hot is unusable at that width, so the options are frequency encoding, smoothed target encoding, hashing, or a library's native categorical handling. Target encoding is the strongest and the most dangerous, because it must be fitted inside each fold or it leaks the label. Use this categorical encoding answer to show the decision, trade-off, and evidence rather than a memorised definition.
Compare it against a genuinely strong prompted baseline on a held-out set drawn from real traffic, using the task metric your product cares about, and run a broad regression set alongside to catch capability lost elsewhere. Falling training loss is evidence that training worked, not that the product improved.
Rank the causes by how often they are to blame and test them in that order: target leakage, a validation split that ignored group or time structure, train-serve skew, then genuine distribution shift. One cheap experiment separates most of them, which is rescoring logged production rows offline with the same artefact.
A training row must contain only feature values that were knowable at its own event timestamp. A naive equality join on entity id attaches whatever the feature table holds now, including values derived from after the event, which leaks the future and produces offline metrics that collapse in production.
Train-serve skew is a mismatch between features used in training and features produced at serving time. Prevent it with shared transformations, point-in-time joins, feature logging, schema checks and monitoring of served feature distributions. It also connects feature store to the point an interviewer is testing.
Target leakage happens when a feature contains information that would not be available at prediction time. Catch it by auditing feature creation times, window boundaries, post-label columns and suspiciously strong validation performance. Use this data leakage answer to show the decision, trade-off, and evidence rather than a memorised definition.
A random train-test split ruins a forecasting model by leaking future observations into training. Use time-based holdouts or rolling-origin backtesting, and build every feature only from data available before the forecast timestamp. Use this time series answer to show the decision, trade-off, and evidence rather than a memorised definition.
They fail on labels rather than models: failures are recorded as repair dates in free-text work orders, the positive class is nearly empty, feature windows leak the outcome, and no baseline was established, so an encouraging first result cannot be trusted or beaten.