What is cross-encoder reranking?

Cross-encoder reranking is a second-stage ranking step that jointly scores each (query, candidate) pair with a full-interaction model – typically after HNSW or hybrid search has produced a shortlist – so relevance benefits from token-level attention that bi-encoder distance alone cannot capture.

How does a cross-encoder differ from a bi-encoder?

Bi-encoders embed the query and each document separately into vectors, then compare those vectors with cosine or another metric. Document vectors can be precomputed and stored in HNSW, which is why first-stage retrieval stays fast at millions of objects. Cross-encoders do not emit reusable document embeddings for ANN. They take the query and one candidate text together as a pair, run joint attention across both, and output a relevance score (often between 0 and 1). That full interaction usually ranks more accurately when the model is well trained, but scoring every corpus item that way is impossible at scale – you would pay a forward pass per document on every request. Hence the pipeline: bi-encoder plus HNSW for recall, cross-encoder only on the survivors for precision.

That division of labor is exactly why reranking exists as a stage rather than as a standalone search.

Where does cross-encoder reranking sit in a retrieval pipeline?

Stage one retrieves tens to hundreds of candidates with vector search, BM25, or hybrid fusion – high recall, acceptable latency. Stage two feeds each candidate (or a chosen text property) plus the query into the cross-encoder and reorders by the new scores. Optional tricks include a different rerank query string than the retrieval query, or raising the first-stage limit so the cross-encoder chooses among a wider pool. Cost and latency scale with shortlist size, not corpus size. For RAG, better top-k ordering matters especially: generators only see a few passages, so promoting the true answer from rank 6 to rank 1 can change the whole response. Cross-encoders shine on subtle domains – legal, medical, scientific – where embedding distance blurs fine distinctions.

Weaviate exposes this pattern through configured reranker integrations.

How do you use cross-encoder-style reranking in Weaviate?

Attach a reranker configuration to the collection, then pass a rerank clause on vector, BM25, or hybrid queries. Weaviate runs the initial search for your limit, scores each returned object with the configured model (often a cross-encoder-class API or local transformers reranker), and returns the same objects reordered by rerank score. You choose which property to score – title versus body can change outcomes – and may override the rerank query text. Important detail: in the default path, limit is both the candidate set and the result count; the reranker does not fetch extra neighbors on its own. Raise limit and slice the top n in your app if you want a larger pool. Expect added latency and model cost; do not treat reranking as a substitute for fixing chunking, embedding mismatch, or an under-tuned HNSW ef.

Misuse shows up as slower queries without better answers – or better answers that users never wait for.

When should you enable cross-encoder reranking – and when skip it?

Enable it when top-k quality is critical, first-stage recall is already solid, and a few extra milliseconds (or more) are acceptable. Skip or shrink it when p99 latency budgets are tight, the shortlist is already excellent, or the corpus is so simple that bi-encoder neighbors suffice. Always A/B with and without the second stage on real queries; measure nDCG or pairwise preference on the final top-k, and watch empty or thin shortlists under filters – a cross-encoder cannot invent missing candidates. Keep the scored property aligned with where relevance lives, and re-evaluate after embedding or chunking changes.

Cross-encoder reranking is joint scoring on an HNSW shortlist – precision after the bi-encoder cast the net. Next, read "What is multi-vector document retrieval?" for another way to enrich document representation, or revisit "What is two-stage reranking?" for the broader pipeline framing.