What is filter selectivity?

Filter selectivity is the fraction of a vector corpus that satisfies a structured predicate – how many objects an allow-list contains relative to the full collection – and it is the primary knob that decides whether filtered HNSW, flat search over matches, or naive post-filtering will meet recall and latency goals.

How do you measure selectivity without confusing the terminology?

The unambiguous quantity is match rate: |allow-list| / |corpus|. A filter that keeps 50% of objects has a high match rate; one that keeps 0.1% has a low match rate and is often called "highly selective" or "restrictive" in everyday engineering talk. Those two phrases point opposite directions on the same axis, so teams should publish numbers, not adjectives. Match rate can be estimated from inverted-index cardinalities, sampled predicates, or offline histograms of properties (stock flags, regions, price bands, tenants). Compound boolean filters multiply uncertainty: AND of rare tags can be far more restrictive than either tag alone. Selectivity is query-local – the same schema field yields different match rates for region = EU versus region = US – so capacity planning needs a distribution of match rates from production logs, not a single magic percentage.

That distribution is why one filtered-search strategy never wins everywhere.

How does match rate change which ANN plan is rational?

When almost everything matches, filtered search resembles unfiltered HNSW: the allow-list check is a small tax, and post-filtering often keeps a full top-k. When almost nothing matches, unfiltered ANN followed by discard frequently returns fewer than k survivors unless you oversample aggressively; pre-filtering with a tiny allow-list is usually better served by flat distance ranking over those few ids than by a graph walk that keeps rejecting neighbors. Mid-range match rates – say a few percent to tens of percent – are where constrained HNSW shines: build the allow-list, walk the graph, admit only legal ids into the result set, and keep non-matching nodes as bridges so connectivity survives. As the allow-list shrinks, constrained HNSW approaches exhaustive behavior over the whole graph for little benefit – that is the regime where a flat-search cutoff pays for itself. As the allow-list grows toward the full corpus, flat scan of matches becomes the expensive mistake.

Selectivity alone is not the whole story; correlation bends the same match rate into different latency curves.

Why can two filters with the same match rate behave differently?

Vector/predicate correlation describes whether matching objects cluster near the query in embedding space. Positive correlation means many of the nearest unfiltered neighbors already pass the filter – post-filter oversampling looks cheap, and sweeping-style walks find legal hits quickly. Negative correlation means the filter removes the neighborhood the query would have preferred – the walk starts in a "wrong" region and must travel far to reach sparse matching pockets, even if the global match rate is moderate. Low match rate plus negative correlation is the hardest combination: empty post-filter shortlists, long constrained walks, and the strongest case for filter-aware traversal strategies. Benchmarks that only vary match rate while randomly assigning labels can understate production pain when price, category, and embedding geometry move together.

Weaviate exposes selectivity-sensitive controls on the filtered HNSW path.

How does Weaviate react to different filter selectivities?

Filtered vector search builds an allow-list from inverted indexes, then runs HNSW so only allow-listed ids enter the result set while edges may still cross non-matching nodes. flatSearchCutoff switches to brute-force over the allow-list when the matching set is small enough that constrained graph search would be a poor deal – the operational expression of "low match rate favors flat." Filter strategies such as sweeping versus ACORN change how the walk spends distance computations under sparse or low-correlation predicates; ACORN is aimed especially at large collections where restrictive filters hurt throughput. Loose filters behave nearly like unfiltered search with a membership check. Operators should tune cutoff and strategy against the match-rate histogram of real queries – tenant isolation, in-stock, geo, and price caps often sit at opposite ends of that histogram in one product.

You only know your selectivity profile if you measure it where users feel it.

How should you operationalize selectivity in evaluation and SLOs?

Bucket production filtered queries by match-rate bands (for example >50%, 10–50%, 1–10%, <1%) and report recall, filled-k, and p95 latency per band. Include both correlated and anti-correlated slices when you build or choose filtered benchmark datasets. Alert when a release shifts traffic toward rarer predicates without retuning cutoff or ef. Do not declare a filter strategy "faster" from a single 50%-match lab filter. After catalog growth, recompute match rates – absolute allow-list size matters for flat-scan cost even when percentages stay flat. Pair selectivity dashboards with property freshness: a stale stock flag changes effective match rate and empties shelves that the index still thinks are full.

Filter selectivity is the match-rate dial on filtered ANN – it chooses among post-filter hope, constrained HNSW, and flat allow-list search. Next, read "What is vector/predicate correlation?" for the geometry that reshapes the same match rate, then "What is oversampling in filtered search?" for how post-filter pipelines try to compensate when selectivity was ignored up front.