Tabular data with modest row counts
For a table of numeric and categorical columns with thousands to low millions of
rows, a gradient-boosted tree ensemble is the model to beat, and beating it with a
neural network takes real effort for a margin that is usually not worth having.
The reasons are structural rather than accidental.
Trees split on one feature at a time at a learned threshold, which is exactly the
shape of most tabular relationships: a rule that changes at an age, a limit, a
tenure. A network has to approximate that step with smooth functions and needs
data to do it. Trees are indifferent to feature scaling, handle skewed
distributions and outliers without transformation, and are largely unaffected by
irrelevant columns, because a useless feature simply never gets chosen for a
split. A network needs every one of those handled by you: scaling, encoding,
imputation, and a decision about how to represent high-cardinality categoricals.
Missing values are native to most boosting implementations and are a design
question in a network.
The effort asymmetry is the part to say out loud in an interview. Boosting gives
you a strong result from defaults in an afternoon, with three hyperparameters
worth tuning. The network needs an architecture, an embedding scheme, a
normalisation strategy, a learning-rate schedule and an early-stopping rule before
it is competitive, and each of those is a place to be wrong. When both land in the
same accuracy band, the one that took a day is the better engineering decision,
and the one that took three weeks has consumed the budget you needed for feature
work that would have moved the metric more than either model.
Deep learning earns its place where the input has structure a network can exploit
and features cannot easily be hand-built: images, audio, long text, sequences with
long-range dependencies, graphs. If your input is a spreadsheet, that argument is
not available to you.
When the explanation has to be exact
There is a real difference between explaining a model and inspecting one, and it
decides some projects.
A logistic regression on a modest feature set gives you a coefficient per
feature. That is the model, not a summary of it, so the statement "this
application was declined because the debt-to-income ratio contributed this much"
is arithmetic anyone can reproduce. A shallow decision tree or a scorecard gives
you the same property as a readable path. Under credit, insurance or clinical
review, where you may have to state the reason for an individual adverse decision
and defend it later, that reproducibility is the requirement.
Attribution methods applied to a network give you something weaker: an
approximation of local behaviour, computed by a method with its own assumptions,
which can disagree between methods on the same prediction. That is genuinely
useful for debugging and it is not the same as a stated reason. It also fails a
practical test that comes up constantly — whether the model behaves monotonically
in a feature that everyone agrees should be monotone. A linear model with a signed
coefficient guarantees it. Boosting libraries can constrain it. A network can be
persuaded towards it and cannot be made to promise it, and "our model sometimes
improves your score when your income falls" is not a sentence you want to defend.
Hard latency and memory budgets on CPU
The last case is arithmetic rather than judgement, so it is the easiest to make
concrete. Suppose the serving budget for the model is five milliseconds inside a
request that must complete in fifty, on the CPU nodes you already run, with no
accelerator and no batching because requests arrive one at a time.
A linear model is a dot product and it fits in that budget with room to spare.
A boosted ensemble of a few hundred shallow trees is a few hundred short branch
sequences and generally fits comfortably. A network deep enough to have been worth
building is a chain of dense matrix multiplications, and on a single CPU core
without a batch to amortise the overheads it may not fit. GPU throughput numbers
do not transfer: the accelerator's advantage is parallelism across a batch, and
serving one request at a time is the case where it helps least.
Memory follows the same logic. A model that has to run inside a mobile
application, on an embedded controller, or in a serverless function with a cold
start you are billed for, is competing against a size limit, and the parameter
count is the whole argument. Quantisation and distillation exist and genuinely
help, but they are additional projects with their own accuracy cost, and choosing
a model that already fits avoids both.
The failure to guard against is the reverse one
The mistake worth naming is not reaching for a network too eagerly. It is
refusing to re-examine the decision. All three arguments above have conditions
attached, and the conditions change.
If your row count grows into the tens of millions, if a raw text or image column
appears alongside the tabular ones, if a genuine sequence structure emerges, or if
you gain the ability to fine-tune a pretrained model rather than train from
scratch, the balance moves. A pretrained model in particular breaks the
small-data argument, because the network is no longer learning representations
from your data alone — which is why an embedding from a pretrained text model fed
as features into a boosted tree is often the best of both, and is worth
volunteering as the answer to a hybrid case.
State the conditions when you give the recommendation. A candidate who says "trees
here, and I would revisit it if we get text columns or pass ten million rows" is
demonstrating the judgement being tested. One who says networks are overkill for
tabular data and stops has given a correct answer for the wrong reason and will
give the same answer when it is wrong.
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).