What is a search trace?
A search trace is a recorded, step-by-step log of exactly what a single HNSW query actually did while it ran — which layers it descended through, which nodes it visited and in what order, what distances it computed, and how the candidate and result queues evolved along the way. It’s a diagnostic tool rather than a part of HNSW‘s core algorithm itself, and its value comes entirely from turning an otherwise opaque, fast-running search into something a person can actually inspect afterward to understand why that specific query behaved the way it did.
What information does a genuinely useful search trace actually need to capture?
At minimum, a useful trace records the sequence of nodes visited during the search, along with each visited node’s distance to the query at the moment it was examined, since this is the core information needed to reconstruct exactly what path the search actually took through the graph. A more complete trace also records which layer each visit happened at, distinguishing the lightweight upper-layer greedy descent from the fuller base-layer search, and captures snapshots of the candidate queue and result queue‘s contents at meaningful points along the way, showing how the search’s sense of “best answers so far” evolved as it progressed. Some traces go further still, recording the specific reason each candidate was either accepted into the result queue or rejected, which is particularly useful for understanding cases where a search’s final results seem surprising or where recall on a specific query turned out unexpectedly poor.
The right level of detail to capture depends entirely on what the trace is actually being used for — a trace meant for casual sanity-checking might only need the final visited-node sequence, while a trace meant for deep debugging of a specific recall problem benefits from capturing essentially everything the search touched along the way, even at the cost of the trace itself becoming considerably larger and more expensive to produce.
Why would anyone actually need this level of detail, given that a search’s final results are already directly observable?
The final results alone only show what a search found — they say nothing about why it found those specific results and not others, or what path it took to get there. If a query’s recall turns out unexpectedly poor, simply looking at the returned results doesn’t reveal whether the search got stuck at a local minimum early on, whether it explored a reasonable portion of the graph but happened to miss a genuinely close point sitting just outside its search radius, or whether the underlying graph itself has a structural weakness — a missing bridge, an unusually sparse local region — that no amount of search-parameter tuning could have fixed for that specific query. A search trace makes these very different underlying causes distinguishable after the fact, by showing exactly what the search actually examined and in what order, rather than leaving an investigator to guess based on the final output alone.
What does producing a search trace actually cost, and why isn’t it enabled by default on every query?
Recording a detailed trace means capturing and storing information about every single step of a search that would otherwise be discarded the moment it was no longer needed — every distance computed, every queue update, every visited node — which adds real memory and, to a lesser extent, computational overhead on top of what an ordinary, untraced search would require. For a production system serving a high volume of queries, enabling detailed tracing on every single request would be wasteful, since the overwhelming majority of queries behave exactly as expected and never need this level of scrutiny after the fact. This is exactly why search tracing is generally treated as an opt-in diagnostic capability, enabled selectively for specific queries under investigation — reproducing a reported problem, running a controlled debugging session, or sampling a small fraction of production traffic for ongoing quality monitoring — rather than as a feature left running continuously across an entire live workload.
How does a search trace actually get used once it’s been captured, in a typical debugging workflow?
A practical debugging session generally starts by reproducing a problematic query with tracing enabled, then examining the resulting trace to see exactly where the search’s behavior diverged from what was expected — did it stop earlier than it should have given the configured efSearch, did it repeatedly revisit the same crowded region of the graph without making real progress toward the query, or did it explore a reasonable path but simply never encounter the specific node that ground truth says should have been found. Comparing a trace from a problematic query against a trace from a similar, well-behaved query often makes the specific point of divergence much easier to spot than trying to reason about the failure purely from the final result set alone.
Having covered what a search trace actually captures and why it’s a valuable, if selectively used, diagnostic tool, the search-failure-mode and query-difficulty glossary pages, under recall and query difficulty, are the natural next stop, since they cover exactly the kinds of underlying problems a search trace is most often used to diagnose. From there, the local-minimum-in-graph-search glossary page shows one of the most common specific patterns a search trace can reveal when a query’s recall turns out unexpectedly poor.