Collaborative and content signals, candidate generation then ranking, cold start, position bias and feedback loops, and offline-online metric divergence.
3 questions in Machine Learning.
Because you cannot score a million-item catalogue per request inside a few tens of milliseconds. Retrieval reduces millions to hundreds with a cheap model judged on recall, then a far more expensive ranker orders those hundreds and is judged on the ordering it produces at the top of the list.
Fall back to content features and segment popularity while behavioural signal is absent, then blend towards collaborative scores as interactions accumulate. New items also need guaranteed exploration impressions, or the already-popular items take all the traffic and nothing can dislodge them.
Almost always the offline evaluation was measured on logs the old ranker produced. Clicks are confounded with position, the training data only contains items the old system chose to show, and a candidate that reproduces the old ordering scores well offline while adding nothing live.