What is a write-ahead log (WAL)?
A write-ahead log (WAL), also called a commit log, is an append-only on-disk record of mutations written before (or as the durable record of) corresponding in-memory changes, so a crash can be repaired by replaying the log instead of losing acknowledged updates that lived only in RAM.
Why do databases write the log before trusting memory?
Fast paths keep working state in memory: LSM memtables, HNSW neighbor arrays, caches. Memory is volatile. If the process dies after updating RAM but before those structures are flushed to immutable segments or snapshots, the only way to honor a successful write response is to have already recorded the intent on stable storage. Append-only logs are cheap: sequential I/O, simple framing, no need to rewrite a whole index per insert. Recovery reads incomplete or unflushed logs, reapplies operations, then continues. Without a WAL, every crash forces a rebuild from raw objects or silent data loss — unacceptable for a production vector database.
The same pattern appears in two flavors inside a typical Weaviate shard.
How does a WAL work for LSM object and inverted stores?
Each write is appended to a bucket’s WAL, then applied to the memtable. When the memtable hits a size or time threshold — or on orderly shutdown — it flushes to an immutable segment; only then can that WAL be marked complete and discarded or recycled. If startup finds an incomplete WAL, recovery replays it into a new segment and marks the WAL done so future boots skip the work. Idle memtables may flush after a quiet period to shrink recovery windows. Deduplication and parallel recovery across buckets cut mean-time-to-recovery when many large WALs exist after a hard crash. The WAL is disaster machinery, not the primary read path: queries read memtable plus segments, not the log file byte by byte under normal operation.
Vector graphs need a related but specialized commit log.
How does an HNSW commit log differ from an LSM WAL?
Building HNSW edges is expensive; the durable log stores the result of those calculations — node placement, connection lists, tombstones — not a replay of every distance comparison. The entire graph can be reconstructed by replaying commit-log entries from scratch, but that grows painful at tens of millions of vectors. Snapshots checkpoint the compacted graph so startup loads the snapshot and only replays post-snapshot log files. Logs rotate at a maximum size; rotated files become compaction and snapshot inputs. An incomplete final record can be truncated back to the last valid entry (self-healing tails). Sync timing may differ from LSM WALs on the same request path, but both participate in crash safety for a single ingest. Tunables such as maximum HNSW log size trade rotation frequency against compaction efficiency and memory during merge.
Weaviate wires these ideas into the request and tenant lifecycle.
How does Weaviate use WALs in practice?
By the time Weaviate returns success for an ingest that touches LSM storage, an LSM WAL entry exists; failure to create it (for example a full disk) fails the request. HNSW mutations ride a parallel commit log. Multi-tenant buckets each keep their own WAL so isolation extends to durability. Delayed WAL flush can batch segment materialization when many tenants deactivate, holding durability in the log until a size threshold triggers larger flushes — fewer tiny I/O storms. Backups flush memtables first so WAL-covered data lands in immutable segments before files are copied. Monitoring exposes LSM WAL recovery counts, durations, and failures so operators see restart cost. None of this replaces backups: a destroyed disk still needs an external restore, but a process crash should not require reimporting the corpus.
Understanding WAL limits keeps expectations honest under load.
What should operators watch when WALs grow?
Large unflushed memtables mean long recovery — prefer idle flushes and adequate disk throughput. Huge HNSW commit-log deltas without recent snapshots mean slow startups — let compactors run and size PERSISTENCE_HNSW_MAX_LOG_SIZE thoughtfully. Do not delete WAL or commit-log files by hand while the node is down unless you accept data loss. After crashes, confirm recovery metrics settle and queries match pre-crash expectations before retiring the incident. Treat “WAL full / disk full” as a hard availability issue: acknowledged durability depends on successful appends.
A write-ahead log is the append-only safety net that makes in-memory LSM and HNSW state restartable. Next, read “What is an incremental checkpoint?” for how snapshots and related checkpoints shrink replay, then “What is tombstone deletion?” for how deletes are recorded in logs and graphs before space is reclaimed.