Site search converts poorly. How would you improve relevance when conversion is the metric you are judged on?

Find out which queries are failing before touching the ranker

The request as stated is unanswerable, because "search converts poorly" is an average over query types that behave nothing alike. So the first work is to segment the query log by volume and outcome, and the segmentation almost always shows that a handful of patterns account for most of the loss.

Four buckets are usually enough to start. Navigational queries name a specific product or brand and should have near-perfect precision, so a failure here is a defect rather than a tuning opportunity. Category queries name a broad need and are judged on the breadth and sensibility of the first screen. Attribute queries combine a category with a constraint, and they fail when the constraint lives in unstructured text rather than in a filterable attribute. Long-tail and misspelled queries are individually tiny and collectively large, and they fail silently because nobody complains about them.

Rank the buckets by lost revenue, which is roughly query volume multiplied by the gap between that bucket's conversion rate and a comparable baseline. That gives you a defensible order of work and, just as importantly, gives you the sentence you will need when someone asks why you did not start with the thing they personally searched for last week.

Signals, and the fact that text match is only one of them

Textual scoring answers whether a document mentions the query terms in a statistically unusual way. It does not know that the matched product is out of stock, has a two-star review average, is discontinued, or is the single most-returned item in the catalogue. A ranking built on text alone will therefore be defensible term by term and commercially poor overall.

The signals worth adding fall into three families. Behavioural signals are derived from the log: click-through rate for a query-document pair, add-to-cart rate, purchase rate, and the inverse signal of a query where a result was clicked and then immediately abandoned. Item quality signals are properties of the product regardless of query: review average and count, return rate, availability, image and attribute completeness. Business signals encode what you would like to happen: margin, promoted stock, own-brand preference, and the depth of inventory you need to clear.

Combining them is where discipline shows. The version that works and can be explained is a two-stage design. A retrieval stage uses text and filters to pull a few hundred candidates cheaply and with high recall, since nothing downstream can rank a document that was never retrieved. A ranking stage then scores those candidates with the fuller feature set, and because the candidate set is small you can afford a learned model there. Keeping the stages separate means a recall problem and a ranking problem never get diagnosed as each other, which is the single most common way search work stalls.

Synonyms are a maintenance commitment, not a data file

Synonyms fix the class of failure where the customer's word and the catalogue's word differ. Nobody searches for "hooded sweatshirt" if they say "hoodie", and no amount of stemming bridges "trainers" and "sneakers". Mining candidates is straightforward: look for query pairs that lead to the same product being purchased, and for query reformulations within a session where the second attempt succeeded.

The trouble is that synonyms are directional and they rot. "Laptop" should retrieve "notebook", but expanding "notebook" to "laptop" fills a stationery search with computers. A seasonal mapping added for one campaign is still live three years later, quietly polluting results. So each entry needs a direction, an owner, a reason recorded at creation, and a review date, and adding one should be accompanied by a check on the queries it affects rather than a deploy and a hope.

Overrides earn their keep, and also their audit trail

Merchandising overrides - pinning, boosting, burying, curated result sets for a named query - are not a failure of the algorithm. A buyer who knows that a particular query drives the launch of a particular range holds information the ranker cannot infer from six days of sparse click data, and refusing to let them express it is how a search team ends up bypassed by a hard-coded landing page.

What makes overrides safe is the surrounding machinery rather than restraint in using them. Every override carries an author, a justification, a start and end date, and a scope limited to the queries it names. There is a report showing how many queries are currently served by an override and what share of search revenue flows through them, because when that share grows past roughly a third you no longer have a ranking system, you have a manually curated directory with a search box in front of it. And the effect of each override is measurable, so a pin that is quietly costing conversion can be removed with evidence rather than argued about on instinct.

Zero results is a page, and someone has to design it

A query returning nothing is the most expensive result in search, because the customer has told you precisely what they want and received a dead end. Treat it as three separate problems. Some zero-result queries are for things you do not sell, and the correct response is an honest one plus a route into a plausible adjacent category, while the query itself becomes a ranked input to the buying team. Some are spelling or language failures, handled by correction with a visible "showing results for" and a way back. Some are over-constrained filter combinations, where the fix is to relax the least important constraint and say which one you relaxed.

The one thing never to do is return an unfiltered grid of popular products with no explanation, because it teaches the customer that your search ignores what they typed. Track the zero-result rate and the rate of exit from those pages as first-class metrics, since neither appears in an overall conversion figure.

Clicks are not relevance, and conversion will lie to you

Here is where an otherwise strong answer usually stops one step short. Optimising on behavioural data creates a loop: the ranker decides what is shown, what is shown determines what is clicked, and what is clicked trains the next ranker. Position bias means the item at rank one accrues clicks for being at rank one, so a mediocre product placed high looks increasingly relevant to your own model. Anything never retrieved accumulates no evidence at all and can never rise, which is why new products and long-tail inventory decay quietly.

Three habits keep the loop honest. Log impressions with their positions, not only clicks, so exposure can be divided out of the numbers rather than ignored. Reserve a small slice of traffic for exploration, where results are deliberately perturbed, and accept that this slice converts slightly worse in exchange for unbiased training data. Maintain a set of human relevance judgements for a fixed sample of queries as a check that is independent of your own ranking, so you can tell an improvement from a tightening loop.

Conversion has one further distortion worth naming out loud. A ranker rewarded purely on conversion learns to surface cheap, familiar, heavily discounted items, because those convert. That can raise the metric while lowering revenue per session and raising the return rate, so the objective you optimise should be closer to margin retained after returns per search session than to raw conversion. Saying this in an interview signals that you understand you are being handed a proxy metric rather than a goal.