What is an ID-to-vector mapping bug?

An ID-to-vector mapping bug is when an object’s external identity (UUID, primary key, or business ID) is paired with the wrong embedding in the vector index – so HNSW faithfully returns "nearest" neighbors that belong to different records than the properties, filters, and application logic assume.

Why does search need a trustworthy ID-to-vector link?

ANN indexes do not search prose; they search vectors keyed by internal document IDs that must resolve back to objects. The product contract is simple: the vector stored for ID X was computed from (or intentionally attached to) the content of X. When that link breaks – vector of article A stored under UUID of article B – every layer above the index lies consistently. Filters still see B’s metadata; the graph neighborhood is A’s semantics; evaluation scripts that join on ID report mysterious recall loss; users click a "similar product" and land on an unrelated SKU. Dimensionality and distance can be perfect. The bug is identity, not geometry.

Most mapping failures are born in the ingest join, not inside the graph walk.

Where do pipelines swap the wrong vector onto an ID?

Classic patterns: embedding a batch of texts in parallel and zipping results back by list position after a shuffle, retry, or partial failure reordered the arrays; joining embedding files to object tables on a non-unique key; off-by-one when CSV columns drift; caching embeddings keyed only by hash of text while two IDs share normalized text; writing vectors to one named-vector slot while reading another; updating object properties without re-vectorizing (or re-vectorizing while a concurrent writer restores an old payload). Client-side races that insert the same UUID twice with different vectors can leave duplicates or last-writer chaos depending on how IDs were assigned. Offline ground-truth builders that re-embed evaluation queries with a different corpus snapshot than the index amplify the same class of error – the "true" neighbor IDs no longer match what the live store holds under those IDs.

Inside the database, identity is often a two-level story: external UUID versus internal doc ID.

How do internal document IDs complicate the picture?

Mutable vector indexes commonly give each graph node an immutable internal document ID separate from the user-facing UUID. Updates may tombstone the old internal ID and insert a new node with a fresh internal ID while keeping the same UUID in the object store. That design is correct when both sides stay in sync. Mapping bugs appear when application code caches internal IDs, when a custom export joins vectors by row number instead of UUID, when deletes leave a brief window where search still surfaces an internal ID the object store can no longer resolve, or when a bring-your-own bulk load attaches vectors to the wrong UUID column. The symptom "no object found for doc id" after delete/update churn is a relative of this failure: the graph still knew an internal ID that the object map no longer owned. Wrong-embedding-dimensionality bugs share overlapping stack traces but fail on length; mapping bugs fail on which row owns a correctly shaped vector.

Weaviate keeps object storage and the vector index aligned when you use its APIs end to end – custom pipelines must do the same.

How does Weaviate keep IDs and vectors paired?

Each Weaviate object has a UUID; the HNSW layer uses internal document IDs under the hood, with updates implemented as delete-and-reinsert so graph mutability stays sound while the UUID remains the stable external key. Prefer letting Weaviate’s configured vectorizer embed from the object’s properties so the vector is derived from the same write that stores the payload. For bring-your-own vectors, pass the vector in the same insert/update call as the properties and UUID – never maintain a side table keyed only by batch index. With named vectors, set target_vector explicitly on query and verify you wrote the embedding into that same name. After deletes, allow search to skip tombstoned IDs rather than caching internal doc IDs in the application. When migrating, export and re-import by UUID with vectors attached per object, and rebuild evaluation sets from the same snapshot.

Proving a mapping bug means checking round-trips, not tuning ef.

How should you detect an ID-to-vector mapping bug?

Fetch objects by UUID with vectors included; re-embed the stored text (or known source fields) independently and compare cosine distance to the stored vector – large gaps on many IDs mean systematic mis-joins. Spot-check search hits: does the returned object’s content match the semantic neighborhood you expect for that query? Compare filter-only retrieval of an ID with vector search that should rank it first for its own embedding. Audit batch jobs for positional zips and non-unique join keys. Distinguish from damaged connectivity (exact search on the stored vectors still finds the right IDs) and from dimensionality bugs (operations error on length before ranking). Fix by correcting the ingest join and reindexing affected objects – graph parameter changes cannot reattach the right embedding to the right ID.

An ID-to-vector mapping bug is a correct distance to the wrong life story. Next, explore the frontier topics starting with "What is a self-tuning ANN parameter?" when you leave failure-mode debugging for adaptive index research, or revisit "What causes a wrong-embedding-dimensionality bug?" when lengths disagree before identity does.