mediumConceptEntryMidSenior
What does regularisation do to a model, and how do L1 and L2 differ in their effect?
5 min readUpdated 2026-08-15
Answer
Short answer
Regularisation adds a penalty on coefficient size to the objective, so each coefficient must buy its own weight with error.L2 shrinks smoothly and spreads weight across correlated features; L1 has a constant gradient towards zero, so it drives coefficients exactly to zero and selects features as a side effect.
Was this answer helpful?
Likely follow-ups
- Why should the intercept normally be left out of the penalty?
- Two features are correlated at 0.98. What does L1 do with them, and what does L2 do?
- How do you choose the penalty strength, and what does the validation curve look like as you sweep it?
- What are the equivalents of L1 and L2 in a gradient-boosted tree?
Related questions
- How do you tell that a model is overfitting, and what do you do about it?mediumAlso on regularisation4 min
- You have a tabular business dataset and a deadline. Which model family do you start with, and why that one?mediumAlso on linear-models4 min
- A flex item with a long URL in it pushes past its container instead of shrinking. Why does flex-shrink not save you?mediumSame kind of round: concept4 min
- A third-party script in the head delays your first paint by a second. What do async, defer and type="module" each change about parsing and execution order?mediumSame kind of round: concept5 min