Part IV — Building HNSW from scratch

These are the topics in this section, each on its own page with a stable path you can bookmark or share.

What topics are covered in this section?

• What does concurrent HNSW construction and querying actually require?
• What does a minimal, correct HNSW implementation look like?
• How should HNSW’s data structures be laid out in memory for performance?
• How do distance calculations and CPU optimization affect HNSW’s speed?
• How should an HNSW index be persisted and recovered safely?

Part IV turns HNSW into working software: a minimal correct implementation, memory layout, distance and CPU optimization, concurrency, and safe persistence and recovery.

Why implement if libraries already exist?

Building a small HNSW clarifies where time and memory go, what races look like, and why snapshots and commit logs matter – knowledge you need even when Weaviate or another engine owns production. This Part is for engineers who will profile, extend, or operate graph indexes, not only call an API.

Five chapters progress from correctness, to layout, to speed, to threads, to durability.

How does this Part connect to Weaviate?

Weaviate’s custom HNSW embodies these concerns at database scale: concurrent mutations, WAL and snapshots, SIMD distances, and careful memory use. Reading Part IV makes Weaviate metrics and failure modes legible. Data-structure, systems, memory, concurrency, and persistence glossaries deepen each chapter.

Prefer understanding before micro-optimizing.

Where next?

Begin with the minimal implementation chapter, then layout and CPU, then concurrency and persistence. Part V contrasts real codebases; Part VII scales beyond one box.

Part IV is HNSW as an engineered artifact. Next, open “What does a minimal, correct HNSW implementation look like?” and build upward through the Part.