Deep Learning Foundations interview questions

Backpropagation, optimisers and learning-rate behaviour, normalisation and regularisation, attention and transformers, and when a neural net is not the answer.

3 questions in Machine Learning.

Machine LearningA 22-minute read on how the topic fits together, before the questions.Machine learning fundamentals58 short answers on one page, for revising rather than studying.

medium3

mediumConcept

What is backpropagation doing, and what does a vanishing gradient look like in practice?

Backpropagation applies the chain rule backwards through a network, reusing each layer's cached activations to get every parameter's gradient in one pass. Gradients vanish when many small factors multiply, and it shows as a stalled loss with early-layer weights barely moving while the last layers still train. It also connects vanishing gradient to the point an interviewer is testing.

4 minmid, senior, staff

mediumConceptScenario

How do you choose an optimiser and a learning-rate schedule?

Choosing an optimizer and learning rate schedule starts with the learning rate, then the optimizer. AdamW is a strong default, SGD with momentum can generalise well when tuned, and warmup plus decay usually matters more than the optimizer name. Use this optimisers answer to show the decision, trade-off, and evidence rather than a memorised definition.

5 minmid, senior, staff

mediumConceptDesign

When is a neural network the wrong choice?

On small and medium tabular data, where gradient-boosted trees usually match it for a fraction of the effort; where a decision must be explained exactly rather than approximately; and where a tight CPU latency or memory budget makes it unservable. The simpler model then wins on every axis that matters.

5 minmid, senior, staff, lead