Top 8 Ranking Interview Questions
Browse 8 real-world technical and behavioral interview questions about Ranking. Review scenarios, edge cases, and architectural best practices.
Why is a recommender built as candidate generation followed by ranking rather than as one model?
Because you cannot score a million-item catalogue per request inside a few tens of milliseconds. Retrieval reduces millions to hundreds with a cheap model judged on recall, then a far more expensive ranker orders those hundreds and is judged on the ordering it produces at the top of the list.
How do you handle cold start for a brand new user and a brand new item?
Fall back to content features and segment popularity while behavioural signal is absent, then blend towards collaborative scores as interactions accumulate. New items also need guaranteed exploration impressions, or the already-popular items take all the traffic and nothing can dislodge them.
Design the home feed for a social network.
Size it first: 200M daily actives opening the feed six times a day is 42k reads/s at peak, while 100M posts fanned out to 200 followers each is 231k feed writes/s, so writes dominate. Push to ordinary accounts, pull for high-follower ones, and store ids so deletes and blocks filter on read.
Site search converts poorly. How would you improve relevance when conversion is the metric you are judged on?
Segment the query log first, then blend textual match with behavioural and business signals into a ranking you can explain, treat synonyms and merchandising overrides as owned content with expiry, and handle zero results as a designed page - while guarding against the click feedback loop.
Your offline ranking metrics improved but engagement dropped in the A/B test. What happened?
Almost always the offline evaluation was measured on logs the old ranker produced. Clicks are confounded with position, the training data only contains items the old system chose to show, and a candidate that reproduces the old ordering scores well offline while adding nothing live.
Write a query returning the second-highest salary in each department, then tell me what it does when two people tie for the top.
To get the second highest salary per department in SQL, rank employees inside each department with DENSE_RANK and filter rank two. DENSE_RANK handles ties where ROW_NUMBER or a global MAX gives the wrong result. Use this window functions answer to show the decision, trade-off, and evidence rather than a memorised definition. It also connects sql queries to the point an interviewer is testing.
Where does a cross-encoder reranker earn the latency it costs you?
It earns it when your gold set shows recall at 50 is much better than recall at 5, because that gap is what reranking converts into answer quality. The cost is a model pass per candidate with nothing precomputable, so the latency budget divided by measured per-batch cost sets the candidate count.
Why is dense retrieval on its own not enough, and how do you combine it with keyword search?
Dense retrieval handles paraphrase and misses literal tokens, while BM25 does the opposite, so you run both and merge the candidate lists. Merging by rank with reciprocal rank fusion is the robust default, because the two engines produce scores on scales that cannot be normalised reliably per query.