Search ranking and relevance

How search results are ordered: retrieval then ranking, text relevance with BM25, business and personal signals, learning to rank, semantic and hybrid search, query understanding, evaluation with NDCG and click metrics, and serving ranking within a latency budget.

Reading is half of it. See this used in a real interview: walk through Design Yelp →

Finding documents that match a query is the easy part; ordering them so the best ones come first is what makes search good. A restaurant search that shows a closed, badly rated place at the top, or a rental search that ignores price, loses users. Search questions in interviews (Yelp, Airbnb, product search) expect you to describe how results are ranked, not just indexed.

Two stages: retrieve, then rank

1. Retrieval (candidate generation): fetch a few hundred or thousand plausible results quickly from an inverted index, a geospatial filter, or a vector index. Cheap scoring only.
2. Ranking: score those candidates with richer features and a model, then return the top results.
3. Optionally re-rank the top results with an even more expensive model and apply business rules (diversity, promotions).

This funnel keeps expensive computation on a small set. It mirrors recommendation systems. See recommendation systems.

Text relevance

BM25, the default in Elasticsearch and Lucene, scores a document higher when:

• query terms appear often in it (with diminishing returns),
• those terms are rare across all documents (inverse document frequency),
• the matching field is short (a match in a title counts more than in a long description).

Improve it with field boosts (title over body), phrase and proximity matching, synonyms, and stemming. See how Elasticsearch works.

Semantic and hybrid search

Text matching misses meaning: "cheap place to eat" should match "affordable diner". Embedding-based retrieval encodes queries and documents as vectors and finds nearest neighbours. Most production systems use hybrid search: combine BM25 and vector results (for example with reciprocal rank fusion), then rank. See vector search and RAG.

Signals beyond text

For local search, distance and "open now" often matter as much as text. See Design Yelp. For rentals, availability and price fit are hard filters plus soft signals. See Design Airbnb.

Learning to rank

Hand-tuned weights stop scaling after a few signals. Learning to rank trains a model (often gradient-boosted trees, or neural networks) on features of (query, document) pairs to predict relevance or the probability of a click or booking. Training labels come from:

• Human judgements of relevance for sampled queries.
• User behaviour: clicks, dwell time, conversions, corrected for position bias (people click the top result partly because it is on top).

Features are computed from the index, a feature store and the request. See feature stores and ML serving.

Query understanding

Before retrieval, interpret the query:

• Spelling correction and normalisation.
• Intent and entity detection: "pizza near me open now" has a category, a location reference and a filter.
• Query expansion with synonyms.
• Autocomplete steering users toward well-formed queries. See tries and autocomplete.

Evaluation

• Offline: metrics on judged queries, such as NDCG (rewards relevant results near the top), precision at k and recall.
• Online: click-through rate, conversion rate, time to first click, abandonment, reformulation rate, measured with A/B tests. See feature flags and A/B testing.
• Watch for feedback loops: popular results get more clicks and stay popular; add exploration and freshness boosts for new items.

Serving within the budget

A search request typically has 100 to 200 ms end to end:

• Retrieval in tens of milliseconds from in-memory indexes, sharded and replicated.
• Feature fetches in parallel, batched model scoring of a few hundred candidates.
• Caching for popular queries (with personalisation applied on top).
• Fallback to BM25 order if the ranking service is slow.

See tail latency.

In the interview

"Retrieve candidates with BM25 plus a geo filter (and vector search for semantic matches), then rank the top 500 with a learning-to-rank model using text score, distance, rating, popularity and personal features, trained on clicks with position bias correction; evaluate with NDCG offline and conversion in A/B tests."

Checklist

• Retrieve cheaply, rank expensively, re-rank the top.
• BM25 with field boosts; hybrid with embeddings for meaning.
• Quality, popularity, context and personal signals.
• Learning to rank from judgements and debiased clicks.
• Query understanding before retrieval.
• NDCG offline, A/B tests online; exploration against feedback loops.
• Latency budget with parallel features, caching and fallbacks.