What is hardware-aware graph topology?
Hardware-aware graph topology is the practice of shaping an ANN proximity graph – its layout in memory or on disk, its degree and layering, and which parts stay hot – so traversal and distance work match real CPU caches, SIMD units, RAM budgets, and storage I/O instead of treating the machine as an abstract RAM tape.
Why does a mathematically fine graph still run slowly on real hardware?
HNSW search is a greedy walk: at each step the engine loads a node’s neighbor list and vectors, computes distances, and jumps to the next candidate. Algorithmically that is logarithmic in the corpus; physically it is a storm of irregular memory accesses. If neighbors that are often visited together live far apart in address space, every hop misses cache and TLB. If vectors are misaligned for wide SIMD loads, distance kernels split cache lines. If the entire multilayer graph plus full-precision vectors must reside in RAM, the topology that maximizes recall may simply not fit – forcing swapping, OOM, or a smaller machine than the graph assumes. Hardware-aware design asks which edges, codes, and node orderings make the same navigability cheaper on the silicon you actually buy.
Awareness shows up at several layers of the stack, from micrometers to disks.
What does hardware awareness look like across the memory hierarchy?
At the CPU level, engines use SIMD (AVX, Neon, and similar) for cosine, dot, and L2 so many dimensions update in one instruction; layout that keeps vectors aligned multiplies those gains. Graph reordering packs frequently co-traversed nodes into contiguous regions so walks reuse cache lines; profile-guided layouts use observed query paths. Compact edge encodings and delta-friendly connection stores shrink the adjacency footprint so more of the graph stays in last-level cache. Quantization (product, scalar, rotational, binary) shrinks vectors in the hot path so RAM holds more of the index, with rescoring against fuller precision for final ranking. At the system level, choosing how disk-backed structures are accessed – for example memory-mapped files versus explicit positioned reads – changes behavior under memory pressure. Disk-oriented topologies go further: keep a small navigational structure in RAM (upper layers or centroids) and page the bulk of vectors or posting lists from SSD with I/O-bounded probes. Disaggregated-memory research partitions graphs and caches representatives so remote fetches overlap with computation. Across all of these, maxConnections and layer policy are no longer only recall knobs – they are also bytes-per-node and random-I/O knobs.
The trade-off is always capability versus cost on a specific machine class.
When should topology follow hardware instead of peak theoretical QPS?
Choose aggressive in-memory HNSW with full vectors when latency targets are tight and RAM is plentiful. Prefer quantized in-memory graphs when you need the same navigability at a fraction of the footprint and can afford rescoring. Prefer disk-centric or hybrid topologies when the working set exceeds economical RAM and slightly higher latency is acceptable. Multi-vector embeddings that spawn many small graph nodes make edge storage a larger share of RAM, so connection packing and degree limits matter more. Under heavy concurrent load, lock and allocator layouts that reduce contention are part of hardware awareness too – a perfect cache layout that serializes all writers still loses. Measure on representative hardware: a topology tuned on a huge LLC machine can thrash on a smaller cloud skew.
Weaviate’s index family is explicitly organized around these hardware realities.
How does Weaviate apply hardware-aware topology ideas?
Default HNSW keeps the navigable graph and vectors in memory for low-latency search; distance kernels use SIMD on supported architectures, and recent releases shrink connection storage (including optimized connection encodings) so the graph’s share of RAM drops without a reindex. Quantization options – especially rotational quantization as a strong default starting point – compress vectors in the HNSW hot path while rescoring protects recall. Flat indexes suit small sets where a full graph is wasteful; dynamic indexes start flat and promote to HNSW when object counts justify the memory. When RAM is the bottleneck at large scale, Weaviate’s HFresh index keeps only a compressed centroid HNSW in memory and stores posting lists on disk with heavy compression, bounding I/O per query via searchProbe and related settings – a topology shaped for disk and memory hierarchy rather than pure in-RAM QPS. Operators can also tune LSM access strategy (mmap versus pread) when virtual-memory behavior under load matters. Plan capacity from vector count, dimensions, maxConnections, and whether the index is in-memory or disk-backed – not from object payload size alone.
Evaluating hardware-aware topology means pairing recall with resource telemetry.
How should you judge whether a topology fits your hardware?
Track RSS, cache-miss proxies, disk IOPS, and p99 latency together with recall@k. If RAM grows linearly with vectors and the box is swapping, move to quantization or HFresh before endlessly raising ef. If CPU is saturated in distance code, confirm SIMD is active and dimensions are healthy; if CPU is idle and disk is busy, the topology is I/O-bound and probe counts matter more than beam width. Re-benchmark after changing compression or index type – hardware-aware gains can look like free speed until rescoring or I/O dominates. Next steps in the glossary connect topology to learning and routing research.
Hardware-aware graph topology is navigability that respects caches, buses, and disks. Next, read "What is learned entry-point selection?" when the frontier shifts from layout to choosing where the walk begins, or revisit resource planning when you need to size Weaviate RAM for an HNSW deployment.