What is bytes-per-vector memory accounting?

Bytes-per-vector memory accounting is the practice of estimating how many bytes of RAM an index spends for each stored vector — not only the raw coordinates, but also graph edges, per-node metadata, and allocator overhead — so you can predict capacity before the working set blows past physical memory.

Why is “dimension times sizeof(float)” not enough?

The vector payload is only the floor. A d-dimensional float32 vector costs 4d bytes, and a quantized or half-precision vector costs less, but HNSW also stores neighbor lists whose size grows with M and with the higher base-layer degree, typically on both ends of each bidirectional edge. Each node carries layer information, offsets or pointers into edge storage, flags, and possibly locks or version fields. Padding and alignment round those records up. Visited-set scratch, heaps, and thread-local buffers add more at query time, and allocators add headers and fragmentation on top of the logical sizes. Quoting only 4d understates the true resident set and is a common reason capacity plans fail when the graph is built.

A useful account splits the cost into a few buckets you can estimate from parameters you already know.

What buckets belong in an HNSW bytes-per-vector estimate?

Start with vector storage: N × d × bytes_per_element, plus any alignment padding between rows. Add graph storage: roughly N × average_degree × ID_width_bytes, remembering base-layer degree is often about 2M and upper layers about M, and that bidirectional links store each edge twice unless the implementation compresses somehow. Add per-node fixed metadata: tens of bytes per node is common once offsets, levels, and flags are aligned. Divide the sum by N for a steady-state bytes-per-vector figure. For a worked sense of scale: 768-d float32 vectors alone are 3072 bytes each; at average degree 40 with 4-byte IDs, edges add on the order of 160 bytes per node before metadata — smaller than the vectors, but still material at tens of millions of points. Raise M, switch to 64-bit IDs, or store full-precision alongside quantized vectors, and the graph and payload terms move a lot. Guidance used for in-memory systems such as Weaviate often speaks in a similar language: plan from vectors plus connections, not vectors alone.

Build time and query time do not share the same peak, which is why accounting must say which moment you are sizing for.

How do peak build and steady-state query memory differ in the account?

Construction may hold candidate lists sized by efConstruction, temporary edge buffers before pruning, and multiple threads each with private scratch — so peak build memory per vector can exceed the finished index. After build, those temporaries release and RSS settles toward the steady-state account above, plus whatever query concurrency needs for heaps and visited structures. Snapshotting or compaction can temporarily double space. Bytes-per-vector for capacity planning should state whether it is finished-index steady state or peak build; conflating them either wastes RAM budget or OOMs during ingest. Quantization changes the vector bucket sharply while leaving edge costs tied to M and ID width — a reminder to recompute the whole account when compression settings change.

Accounting only helps if it is checked against what the process actually maps.

How should you validate bytes-per-vector numbers against a real HNSW process?

Compare the analytic estimate to measured RSS or proportional set size after a cold build and after a warm query load, then divide by N. If measured bytes-per-vector are far above the estimate, look for allocator fragmentation, per-node heap allocations instead of contiguous arenas, debug metadata, or duplicate vector copies (for example raw plus quantized). If they are far below, you may be measuring before the graph is fully loaded or missing memory-mapped regions in the tool’s view. Track the figure as N, M, and dimensionality evolve so regressions in layout show up as a rising bytes-per-vector trend, not only as an OOM at the worst moment. Good accounting turns HNSW memory from a surprise into a parameter you can budget beside recall and latency.

Bytes-per-vector memory accounting prices each point as vectors plus edges plus metadata, which is how you size RAM for HNSW before you build. From here, the pages on graph storage overhead and vector storage overhead split those buckets, peak build memory and steady-state memory name the two planning horizons, neighbor-ID width and M explain the edge term, and the chapter on laying out HNSW in memory shows how layout choices move the measured number.