What is Jaccard similarity?

Jaccard similarity measures how much two sets overlap by comparing the size of their intersection to the size of their union, which makes it the natural choice for comparing sets of items — tags, categories, or shared neighbors — rather than fixed-length numeric vectors.

How is Jaccard similarity actually calculated?

Take the items the two sets have in common, count how many there are, and divide that count by the total number of distinct items across both sets combined. Two identical sets score a Jaccard similarity of 1, since their intersection and union are the same set, and two sets that share nothing at all score 0, since their intersection is empty. Everything in between reflects how much overlap exists relative to how much total variety the two sets contain together, so two large sets that share a handful of items score lower than two small sets that share those same few items, because the union in the first case is so much larger.

Why doesn’t Jaccard similarity apply to the same fixed-length vectors as the other distance functions in this glossary?

Euclidean distance, cosine similarity, and the other distance functions covered elsewhere in this glossary all assume both inputs are vectors of the same fixed length, where each position lines up meaningfully with the same position in the other vector. Sets don’t work that way — two sets of tags or categories can be completely different sizes, and there’s no meaningful notion of “position three” in one set lining up with “position three” in another. Jaccard similarity was built specifically for this case, comparing sets by what they contain rather than by aligning corresponding positions, which is why it shows up in contexts like comparing tag sets or the recommended-neighbor lists produced during evaluation, covered in this site’s page on recall as an evaluation metric, rather than in the core distance calculations used while actually searching an HNSW graph.

Where does Jaccard similarity actually show up in an HNSW-based system?

It rarely appears as the distance function driving the graph search itself, since HNSW is built around comparing dense numeric vectors rather than sets. Instead, Jaccard similarity tends to show up around the edges of such a system: comparing the sets of results returned by two different search runs to see how much they agree, comparing tag or category sets attached to items as a filtering or reranking signal, or evaluating how similar the neighbor sets found by an approximate search are to the neighbor sets a brute-force search would have found. In each of these cases, what’s being compared is fundamentally a set of discrete items rather than a point in continuous vector space, which is exactly the situation Jaccard similarity was designed for.

Jaccard similarity fills a different role than the vector distance functions covered elsewhere in this glossary, suited to sets rather than fixed-length numeric vectors. From here, the page on recall as an evaluation metric shows one concrete place this kind of set comparison gets used, and the page on angular distance continues this glossary’s coverage of the vector-based distance functions.