Linear and logistic models, trees and gradient boosting, regularisation, and choosing a family from the data and the constraints rather than by habit.
3 questions in Machine Learning.
Start with gradient-boosted trees and keep a regularised linear model as the baseline. Boosting dominates business tables for structural reasons, not fashionable ones. It ignores feature scale, splits on thresholds rather than slopes, absorbs missing values, and finds interactions you never named.
Regularisation adds a penalty on coefficient size to the objective, so each coefficient must buy its own weight with error. L2 shrinks smoothly and spreads weight across correlated features; L1 has a constant gradient towards zero, so it drives coefficients exactly to zero and selects features as a side effect.
Rank the causes by how often they are to blame and test them in that order: target leakage, a validation split that ignored group or time structure, train-serve skew, then genuine distribution shift. One cheap experiment separates most of them, which is rescoring logged production rows offline with the same artefact.