What is a thread, in a concurrent system?

A thread, in a concurrent system, is an independently scheduled stream of execution inside a process — with its own program counter, registers, and stack — that can run in parallel with other threads while still sharing the same address space, heap, and open resources.

How does a thread differ from a process, and why does that matter?

A process owns an isolated virtual address space and is the unit the operating system uses to separate programs from each other. Creating a new process is relatively heavy: new page tables, new handles, expensive context switches across protection boundaries. A thread lives inside an existing process and reuses that process’s memory map, so spawning another thread is cheaper and communication can be as direct as reading and writing shared data structures. The trade-off is immediate: because threads share memory by default, one thread’s write is visible to others, and without synchronization those concurrent reads and writes can race. Processes isolate faults and data by default; threads share for speed and must add locks, atomics, or disciplined ownership on top. Concurrent HNSW systems almost always choose threads (or an equivalent task runtime on top of threads) so many searches or insertions can touch one in-memory graph without copying the index per worker.

That shared-graph model is exactly how production HNSW query serving and multithreaded construction scale out on a single machine.

How do HNSW implementations typically use threads?

A single search is usually one thread walking the graph with its own candidate heap, result heap, and visited set — private structures that need no sharing. Throughput comes from running many such searches at once, each on its own thread, against a graph that is read-mostly while queries run. Construction is different: many threads may insert nodes concurrently, each performing searches and then splicing neighbor lists into the shared graph, which requires careful synchronization so two insertions do not corrupt the same node’s adjacency data. Batch query APIs often map one batch onto a pool of worker threads. In-memory vector databases such as Weaviate rely on this pattern: the HNSW index lives once in shared memory, and concurrent request handlers are threads (or goroutines multiplexed onto threads) that borrow CPU time to traverse it. More threads raise potential QPS only while there is idle core time and unused memory bandwidth; past that point they mostly add contention.

Sharing an address space also means sharing the memory hierarchy’s bottlenecks, which is why “just add threads” eventually stops helping.

What limits how many threads help an HNSW workload?

Each active search thread issues its own stream of cache misses and pointer-chasing loads. Together they compete for L3, memory controllers, and — on NUMA machines — interconnect bandwidth. When those shared resources saturate, adding threads increases CPU utilization without a matching rise in QPS, and latency percentiles can worsen because every query waits longer in line for DRAM. False sharing on packed counters, lock contention on global structures, and allocator contention during build are further thread-scaling traps. SIMD still helps inside each thread’s distance kernel; it does not create more memory bandwidth for the pool. Sensible scaling therefore sizes the thread pool to cores (or to a measured bandwidth cliff), pins threads near their data on NUMA hosts, and keeps per-query state on the stack or in thread-local storage so workers do not bounce cache lines among themselves unnecessarily.

Correctness under concurrency is the other half of the story: threads make races possible the moment any write shares a location with another thread’s read or write.

What must be private per thread versus protected when shared in HNSW?

Per-query heaps, visited sets, and scratch vectors should be private so threads never synchronize on the search hot path. The graph’s neighbor lists and vector storage are shared: during read-only querying they can be traversed without locks if the implementation guarantees no writers, or with reader/writer locks if inserts happen concurrently. During concurrent insertion, updates to a node’s edges need mutual exclusion or carefully designed lock-free protocols so readers never see torn lists. Global metadata — entry point, maximum level, node counters — is shared state that must be updated atomically or under locks. Understanding threads as “private control, shared data” is the mental model behind every later glossary entry on mutexes, atomics, and contention: the thread is the actor; those mechanisms are how actors coordinate when their shared HNSW structures overlap.

A thread is the unit of concurrent execution that lets many HNSW searches share one in-memory graph — and that forces explicit rules about what stays private versus what must be synchronized. From here, the concurrency glossary pages on parallel query serving and multithreaded construction show the two main HNSW thread patterns, atomics and mutexes cover coordination tools, CPU affinity and NUMA explain placement, and the chapter on concurrent HNSW construction and querying goes deeper on the protocols those threads must obey.