The modelling half of an ML loop: supervised and unsupervised methods, features and leakage, metric choice, imbalance, deep learning foundations, forecasting and ranking.
24 published across 8 topics.
Machine learning fundamentals58 short answers on one page, for revising rather than studying.Linear and logistic models, trees and gradient boosting, regularisation, and choosing a family from the data and the constraints rather than by habit.
Start with gradient-boosted trees and keep a regularised linear model as the baseline. Boosting dominates business tables for structural reasons, not fashionable ones. It ignores feature scale, splits on thresholds rather than slopes, absorbs missing values, and finds interactions you never named.
Rank the causes by how often they are to blame and test them in that order: target leakage, a validation split that ignored group or time structure, train-serve skew, then genuine distribution shift. One cheap experiment separates most of them, which is rescoring logged production rows offline with the same artefact.
Regularisation adds a penalty on coefficient size to the objective, so each coefficient must buy its own weight with error. L2 shrinks smoothly and spreads weight across correlated features; L1 has a constant gradient towards zero, so it drives coefficients exactly to zero and selects features as a side effect.
Encoding, scaling and interaction terms, target and temporal leakage, train-serve skew, and the features that quietly encode the label.
One-hot is unusable at that width, so the options are frequency encoding, smoothed target encoding, hashing, or a library's native categorical handling. Target encoding is the strongest and the most dangerous, because it must be fitted inside each fold or it leaks the label.
Skew is any difference between the feature values a model trained on and the ones it receives at request time. It comes from two implementations of one transformation, joins that ignore the as-of timestamp, and features unavailable in a request. Prevent it with shared code, as-of joins and served-vector logging.
Target leakage is a feature carrying information that would not exist at prediction time, so the model is partly reading the answer. The two forms are a column written after the label event and an aggregate over a window including it. You catch it by auditing when each column is written.
Cross-validation that respects structure, ROC versus precision-recall, calibration, threshold selection, and tying a metric to a business decision.
Build the split to imitate the gap between training and deployment. Group all of an entity's rows onto one side when rows repeat per entity, split forward in time for anything temporal with an embargo covering the label window, and treat a plain random shuffle as the default that silently inflates almost every score.
ROC-AUC hides false positives under imbalance because the false-positive rate divides by a huge negative count, so thousands of wrong alerts barely move it. Precision-recall exposes that honestly but is anchored to the base rate, so its numbers cannot be compared once prevalence moves.
A model is calibrated when among all cases it scores 0.30, close to 30% are positive. Most classifiers emit a score that ranks well and is not a probability, so any decision built from expected value is wrong by an amount nobody measures. Fit isotonic or Platt scaling on a held-out set to repair it.
Fraud and failure-scale imbalance, resampling and class weights, why accuracy is meaningless here, and evaluating at the operating point you will run.
Work backwards from the review capacity. Fix the alert budget the team can clear, evaluate precision and recall at exactly that budget on data at the real prevalence, define the label and its arrival delay before modelling, and never report accuracy, which a do-nothing model scores 99.95% on.
Derive it from the two error costs. Acting is worthwhile when its expected cost falls below the cost of inaction, which gives a threshold of the false-positive cost over the sum of both. With calibrated probabilities that threshold is fixed as prevalence moves; under a fixed alert budget it must move.
None of them adds information. Weights change the loss without touching the data, oversampling duplicates rows and inflates any score computed after it, undersampling discards real observations, and SMOTE interpolates implausible points. All shift the implied prior, so the scores need recalibrating.
Clustering and its evaluation, dimensionality reduction, density and isolation methods, and detecting anomalies with no labels to learn from.
Model what normal looks like and score deviation from it, using distance or density methods, an isolation forest, or reconstruction error from an autoencoder. Then set the threshold from how many alerts the team can investigate per day, because an unlabelled score distribution cannot tell you where to cut.
Combine internal geometry measures such as silhouette with stability under resampling, then accept that neither picks k for you. A clustering is good when the segments are separable, reproducible on a resampled dataset, and distinct on a variable the business will act on.
It helps when features are genuinely redundant, when a downstream method needs dense low-dimensional input, or when you must compress for cost. It hides the problem when PCA's linearity assumption discards the signal, and when a t-SNE or UMAP picture is used as evidence for a modelling decision it cannot support.
Backpropagation, optimisers and learning-rate behaviour, normalisation and regularisation, attention and transformers, and when a neural net is not the answer.
Backpropagation applies the chain rule backwards through a network, reusing each layer's cached activations to get every parameter's gradient in one pass. Gradients vanish when many small factors multiply, and it shows as a stalled loss with early-layer weights barely moving while the last layers still train.
Start with AdamW because it needs less tuning, and consider SGD with momentum where you have the budget to tune it and want its generalisation behaviour. The schedule matters more than the choice: a short warmup, then decay, and read the loss curve to tell a rate that is too high from a model that has stopped learning.
On small and medium tabular data, where gradient-boosted trees usually match it for a fraction of the effort; where a decision must be explained exactly rather than approximately; and where a tight CPU latency or memory budget makes it unservable. The simpler model then wins on every axis that matters.
Stationarity and seasonality, backtesting without lookahead, horizon and hierarchy, and why a random split ruins a forecasting model.
Forecast at whichever levels have signal, then reconcile so the numbers are coherent by construction. Bottom-up preserves detail but accumulates noise, top-down is stable but allocates by an assumed proportion, and optimal reconciliation projects independent forecasts onto the set that adds up.
Plot the series first, then use a decomposition and an ADF or KPSS test to separate trend from seasonality from noise. Difference or transform away a trend, use Fourier terms when several seasonal periods coexist, and treat holidays and calendar structure as features rather than outliers.
Because it puts future observations in the training set, so the model is scored on interpolating between points it has already seen rather than on predicting forward. The correct evaluation is a rolling-origin backtest, and every feature must be computable from data available strictly before the timestamp it describes.
Collaborative and content signals, candidate generation then ranking, cold start, position bias and feedback loops, and offline-online metric divergence.
Because you cannot score a million-item catalogue per request inside a few tens of milliseconds. Retrieval reduces millions to hundreds with a cheap model judged on recall, then a far more expensive ranker orders those hundreds and is judged on the ordering it produces at the top of the list.
Fall back to content features and segment popularity while behavioural signal is absent, then blend towards collaborative scores as interactions accumulate. New items also need guaranteed exploration impressions, or the already-popular items take all the traffic and nothing can dislodge them.
Almost always the offline evaluation was measured on logs the old ranker produced. Clicks are confounded with position, the training data only contains items the old system chose to show, and a candidate that reproduces the old ordering scores well offline while adding nothing live.