mediumConceptScenarioEntryMidSeniorStaff
Why does a random train-test split ruin a forecasting model?
A random train-test split ruins a forecasting model by leaking future observations into training. Use time-based holdouts or rolling-origin backtesting, and build every feature only from data available before the forecast timestamp. Use this time series answer to show the decision, trade-off, and evidence rather than a memorised definition.
5 min readUpdated 2026-08-15
Answer
Short answer
A random split is wrong for forecasting because it lets the model train on future periods and validate on past periods, turning prediction into interpolation. Use a time-based split or rolling-origin backtest: train only on data before the cutoff, forecast the next horizon, score it, then roll the cutoff forward.
Was this answer helpful?
Likely follow-ups
- Should the training window expand or slide as the origin rolls forward, and when does it matter?
- You have a target-encoded promotion flag. How do you compute it without lookahead?
- How do you choose the number of origins and the size of each test window?
- Your feature is available in the warehouse at 06:00 but describes yesterday. Is that safe?
Related questions
- How do you tell that a model is overfitting, and what do you do about it?mediumAlso on cross-validation and data-leakage4 min
- How would you design a validation strategy that respects the structure of the data you have?hardAlso on cross-validation and data-leakage5 min
- You forecast every store and also the national total, and the store forecasts do not sum to it. What do you do?hardAlso on forecasting and time-series5 min
- A churn model scores 0.94 AUC offline and barely beats the baseline in production. One of its strongest features is populated by a process that runs after the customer has already churned. Explain what happened and how you would fix it.mediumAlso on data-leakage5 min