Systems and hardware concepts

Systems and hardware concepts explain why HNSW latency is often a memory-hierarchy story: caches, TLBs, SIMD distance kernels, NUMA, and the locks that protect concurrent graph updates.
Created: Updated: 2 min read

These are the topics in this section, each on its own page with a stable path you can bookmark or share.

What topics are covered in this section?

Systems and hardware concepts explain why HNSW latency is often a memory-hierarchy story: caches, TLBs, SIMD distance kernels, NUMA, and the locks that protect concurrent graph updates.

Where does query time actually go on a machine?

Distance kernels stream vector bytes through caches and SIMD units; pointer chasing across adjacency lists stresses latency and TLBs. Cache lines, cache misses, locality of reference, memory bandwidth, and main-memory latency name the bottlenecks. The roofline model frames whether you are compute-bound or memory-bound. Registers, branch misprediction, and vectorized distance computation refine micro-optimizations. On multi-socket boxes, NUMA placement can dominate if the graph and vectors sit far from the CPU running the walk.

Hardware vocabulary turns “HNSW is slow” into a measurable diagnosis.

How do concurrency primitives fit?

Threads, atomics, mutexes, reader/writer locks, false sharing, lock ordering, deadlocks, and contention appear whenever construction and queries overlap. HNSW hubs can become hot spots under write-heavy loads. Weaviate’s production engine invests heavily in concurrent safe updates and SIMD-friendly distances; understanding these terms helps interpret CPU profiles and why raising thread count does not always raise QPS.

Part IV and Part VII expand these ideas into implementation and accelerators.

Start with cache line, SIMD, pointer chasing, and contention pages, then memory-layout and concurrency hubs. For disk and GPU paths, continue into Part VII.

This section connects HNSW to real silicon. Next, read “What is SIMD (single instruction, multiple data)?” or “What is a cache miss?”, then Part IV’s CPU-optimization chapter.