What is full-precision reranking?

Full-precision reranking is a second scoring pass that recomputes distances between the query and a shortlist of candidates using the original high-precision vectors – usually float32 – after an HNSW search has already ranked those candidates with cheaper compressed or reduced-precision distances.

Why does compressed HNSW search need a second pass at all?

Scalar quantization, product quantization, int8 codes, and even FP16/BF16 storage speed up the many distance evaluations inside graph traversal by touching fewer bytes and using leaner arithmetic. Those approximate scores are good enough to decide which neighbors to expand next, but they can mis-order two vectors that are nearly tied under the true metric. If you returned the compressed ranking as the final top-k, recall would absorb every transposition quantization error introduced. Reranking keeps the fast approximate walk for exploration, then spends a small, fixed budget of exact (or much more accurate) comparisons only on the survivors. The graph still does the hard work of not scanning the whole collection; full precision only polishes the shortlist.

That pattern only works if the shortlist is wide enough to catch the true neighbors despite noisy scores.

How do over-fetching and rescoring fit together?

Typical pipelines ask HNSW for more candidates than the user-facing k – for example retrieve 100 or 200 IDs under quantized distance when the API asks for top-10. Each of those candidates is then compared to the query with full-precision vectors loaded from a sidecar store, disk page, or cache reserved for rescoring. Sorting that shortlist by exact distance yields the final results. The over-fetch depth (sometimes called a rescore limit) is the knob: too shallow and true neighbors excluded by compression never get a chance to be corrected; too deep and you burn latency and I/O re-reading large float vectors. Raising efSearch also widens exploration under approximate scores and pairs naturally with reranking. Vector databases such as Weaviate build this over-fetch-then-rescore behavior into several quantized HNSW modes so recall stays close to uncompressed search while vectors in the hot path stay small.

Where the full-precision vectors live determines whether reranking stays cheap.

What storage layouts make full-precision reranking practical?

One design keeps compressed codes in RAM for traversal and leaves float32 vectors on disk or in a cooler tier, reading only the shortlist – ideal when RAM is tight and a few dozen random reads beat holding every float in memory. Another keeps a full-precision cache of hot IDs or stores both representations in memory when capacity allows, trading RAM for stable latency. A third uses reduced floats (FP16) for the hot path and float32 only for the final sort. In all cases the query vector should match the metric assumptions of the exact pass (same normalization, same distance definition) or the rescoring will “correct” toward the wrong geometry. Reranking does not fix a graph built under a mismatched metric; it only reorders candidates the walk already found.

Used well, reranking is how you buy compression without surrendering ranking quality.

How should you tune full-precision reranking for production HNSW?

Plot recall versus latency while sweeping over-fetch depth and efSearch together – the right pair is an operating point, not two independent defaults. Measure the fraction of query time spent in the exact pass; if it dominates, shrink the shortlist, keep full-precision vectors warmer, or ease compression. Confirm that deleted or tombstoned IDs cannot appear in the shortlist without a validity check. When approximate distance computation is already very faithful, a tiny rescore limit may suffice; when you use aggressive PQ or binary codes, budget a deeper shortlist. Full-precision reranking is the safety net under two-stage retrieval: explore cheaply, decide expensively, but only for a few dozen vectors.

Full-precision reranking reorders an HNSW shortlist with original high-accuracy distances so compressed search can stay fast without locking in quantization mistakes. From here, two-stage retrieval (quantize then rerank) frames the whole pipeline, approximate distance computation and quantization error explain what the first pass gets wrong, scalar and product quantization are the usual reasons you need the second pass, and oversampling in filtered search shows a related “fetch more, then refine” idea in another chapter.