Read the columns before choosing anything
A business table is heterogeneous in a way that an image or a waveform is not. One column is a monetary amount spanning four orders of magnitude, the next is a count that is zero for most rows, the next is a postcode with thousands of levels, the next is a timestamp, and several are missing for reasons that are themselves informative. There is no translation invariance to exploit, no spatial locality, no shared structure across columns. The columns are not samples of one signal, they are different measurements that happen to describe the same entity.
That description is the argument for trees, and it is worth making in those terms because it is falsifiable. Everything a gradient-boosted ensemble does well maps onto a property of that table.
Why boosting fits heterogeneous columns structurally
A tree splits on amount > 4200, so the units of amount are irrelevant and any monotone transformation of it produces the same split structure. You do not standardise, you do not take logs, and you do not lose a day discovering that one unscaled column dominated the gradient. A linear model or a neural net requires you to get that scaling right, and requires you to guess the shape of the relationship: a linear term cannot express "risk is flat until the balance passes the credit limit and then jumps", which is exactly the shape that business rules and human behaviour produce. Trees fit sharp thresholds natively because a threshold is the only thing they can fit.
Missing values are handled inside the split rather than before it. Modern boosting implementations learn a default direction for missing rows at each node, so the absence of a value becomes a signal that the model can use, instead of an imputed number that pretends to be a measurement.
Interactions are where the gap is widest, and it is worth quantifying. Suppose twelve features. To let a linear model express every pairwise interaction you must construct C(12,2) = 66 product terms; every three-way interaction adds C(12,3) = 220 more, taking you to 286 engineered columns before you have looked at the data. A tree of depth six expresses interactions up to order six along every root-to-leaf path without you naming a single one, because each successive split is conditional on the splits above it. That is not a small convenience. It is most of the feature engineering you would otherwise have done by hand.
The baseline exists to be measured against
Fit a regularised linear or logistic model first, on the same folds, and record its score. Its job is not to win. Its job is to tell you what the boosted model is worth. If logistic regression reaches 0.79 AUC and the boosted ensemble reaches 0.81, the interesting question is no longer which model to ship but whether the extra two points justify a second artefact to monitor, retrain and explain. Often they do not, and the linear model has coefficients you can put in front of a credit committee.
The baseline also functions as a leakage alarm. A linear model scoring 0.99 on a problem nobody has ever solved is telling you that one column encodes the label, and it tells you which column, because you can read the coefficients.
When something else earns the slot
Neural networks earn their place when the input has structure the network can exploit and a tree cannot: raw text, images, audio, long sequences, or a very high-cardinality embedding-friendly space such as user and item identifiers in a recommender. A hybrid is common and sensible, where a transformer encodes the free-text column into a vector and the boosted model consumes that vector alongside the numeric columns.
Constraints can also decide it. Inference latency in the low microseconds may rule out a deep ensemble in favour of a scored linear model or a small tree. Hard monotonicity requirements are more easily satisfied by a model that supports monotonic constraints on named features, which the mainstream boosting libraries do. A requirement to explain individual decisions in a legally defensible way pushes towards a model whose mechanism a non-specialist can follow.
Mistaking a habit for a reason
The weak version of this answer is "gradient boosting is what everyone uses on tabular data". That is true and it scores nothing, because the interviewer cannot distinguish it from a candidate who has read one blog post. The strong version names the mechanism: scale invariance, native missing-value handling, non-smooth thresholds, conditional interactions learned for free. A candidate who has those four reasons can also predict where the default fails, which is the actual test. Very wide and very sparse inputs, genuinely smooth physical relationships, extrapolation beyond the training range where a tree returns a constant, and any input that is a sequence rather than a row.
The other half of the trap runs the opposite way. A candidate who reaches for a neural network on a 50,000-row table with thirty columns has usually chosen the family for its interest value rather than the data's shape, and will spend the deadline on scaling, initialisation and learning-rate schedules that a boosted model would not have needed.
Choose the family from properties of the columns you can point at, and always carry a baseline whose only purpose is to tell you what the complicated model bought you.
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).