What is candidate generation and ranking?

Candidate generation and ranking is the two-stage pattern where a cheap, high-recall retriever – often HNSW over embeddings – first pulls a shortlist from a huge catalog, then a slower, more precise scorer reorders that shortlist into the final top results.

Why split retrieval into generation and ranking at all?

Scoring every item in a million- or billion-scale inventory with a rich model is too slow for interactive latency. Candidate generation (also called first-stage retrieval) answers a different question: which few hundred or few thousand items are even plausible? It optimizes for recall and speed – approximate nearest neighbors via HNSW, lexical BM25, trending lists, or several of these in parallel – accepting that the shortlist will contain noise. Ranking (second stage) then spends compute only on that shortlist: cross features between user and item, calibrated click models, business boosts, diversity penalties. Precision becomes affordable because the candidate set is small. If generation misses a relevant item, ranking can never surface it – so teams measure generation with Recall@K and ranking with NDCG or pairwise accuracy on the shortlist that generation actually produced.

Those two jobs map cleanly onto different model and index choices.

What does candidate generation look like when HNSW is the funnel?

Item (or document) embeddings are precomputed and indexed in an HNSW graph. At request time you embed the user, query, or context once, then run ANN search for a large k – often hundreds to thousands. Multi-lane generation is common: one HNSW space for collaborative-style vectors, another for content embeddings, plus a keyword lane; unions or reciprocal-rank fusion merge them before ranking. Hard filters (in stock, region, age rating) usually apply during or immediately after generation so invalid items never waste ranker capacity. Generation parameters trade recall for latency: higher query ef and larger k feed the ranker better candidates at higher cost. The generation stage should stay sublinear and cache-friendly; it is the HNSW walk (and sibling retrievers), not a full catalog scan.

Ranking then consumes that shortlist with signals generation could not afford.

What does the ranking stage do that ANN alone cannot?

A bi-encoder or two-tower distance is a single similarity number. Rankers can compare query and candidate jointly (cross-encoders), fold in freshness, price, margin, personalization features, and apply policy rules. Lightweight score rankers reweight generation scores; metadata rankers boost or bury by attributes; heavier neural rankers rescore each pair. Diversity methods such as maximal marginal relevance (MMR) are a form of reordering: retrieve a larger candidate pool, then iteratively pick items that stay relevant while differing from what was already chosen – Weaviate exposes this as query-time diversity selection on top of vector or hybrid search. Hybrid fusion inside Weaviate (vector plus BM25 combined by ranked or relative-score fusion) is itself a mid-stage blend before an optional external rerank module. The shared idea: generate broadly, then spend precision budget where it matters.

Weaviate often owns generation and light reordering; heavier rankers sit beside it.

How do you wire this pattern with Weaviate in practice?

Use Weaviate HNSW (and optional hybrid search) to produce the candidate set with a generous limit, attaching filters so the allow-list matches product rules. Optionally apply built-in MMR or a configured rerank module to reshape that set. Pass the surviving IDs and features to your application ranker for final ordering, then return the top slice to the UI or to a RAG prompt. Tune generation and ranking separately: raise ef or candidate limit when labeled Recall@K is low; improve the ranker when generation recall is fine but user metrics still lag. After embedding-model changes, re-check both stages – a new geometry can preserve ANN neighbors while breaking a ranker trained on old scores, or the reverse.

Candidate generation and ranking is how large-scale search and recommenders stay both fast and sharp – HNSW casts a wide net; ranking chooses the catch. Next, read "What is two-stage reranking?" for the scoring hand-off in more detail, or "What is cross-encoder reranking?" when the second stage is a joint neural scorer.