What is RDMA (remote direct memory access)?

RDMA (remote direct memory access) is a networking capability that lets one machine read or write another machine’s memory with minimal involvement from the remote CPU and OS network stack, cutting latency and CPU overhead compared with ordinary request/response RPCs over TCP.

How does RDMA differ from ordinary network messaging?

In a typical distributed search call, the client or coordinator sends a message, the remote kernel wakes, copies buffers, a userspace server handles the request, and a reply travels back – every hop burns CPU and adds microseconds. RDMA registers memory regions with a specialized NIC so a local process can issue one-sided operations (for example RDMA READ) that pull bytes straight from remote DRAM into local buffers, or WRITE into remote memory, without the remote application thread servicing each transfer. Two-sided RDMA still involves remote CPU for verbs completion, but the data path stays far lighter than a full HTTP or gRPC stack. Transports such as InfiniBand and RoCE (RDMA over Converged Ethernet) provide the reliable, in-order delivery these verbs assume. The result for systems software is lower tail latency and more host cycles left for distance calculations instead of packet processing – attractive whenever ANN search moves large vector or graph chunks across the network often.

That property matters because classic graph ANN fights the network differently than bulk transfers do.

Why is HNSW-style search awkward on naïve RDMA?

HNSW search is pointer-chasing: each hop needs neighbor lists and vectors that depend on the previous distance comparisons. If every hop triggers a separate remote READ, round-trip count explodes and RDMA’s bandwidth advantage is wasted on tiny, dependent messages. Research systems that put HNSW on disaggregated memory therefore redesign layout for RDMA: partition the graph into contiguous clusters that can be fetched in fewer large READs, keep a small local routing or representative index so the compute node knows which partitions to pull, reserve overflow regions so inserts do not shatter contiguity, and pipeline READs with distance compute so the NIC and CPU overlap. One-sided designs intentionally bypass memory-node CPUs during query so the memory pool scales as a dumb byte store; RPC-over-RDMA baselines that still run full search on the memory side reintroduce remote CPU as the bottleneck. The lesson is that RDMA is not a drop-in speedup for unchanged HNSW – the index and I/O schedule must become RDMA-aware.

Most production vector databases never expose these verbs to operators.

Where does RDMA show up relative to systems like Weaviate?

Weaviate and similar service-oriented vector databases typically speak ordinary TCP/HTTP/gRPC between clients, coordinators, and shards. Latency work there focuses on region placement, in-process HNSW, quantization, and shard fan-out – not registering remote memory regions. RDMA appears in HPC clusters, some cloud bare-metal offerings, and research engines that treat memory as a pooled fabric resource. Operators of Weaviate still feel “network cost” as RPC fan-out and object payload fetch; they do not configure queue pairs or memory keys. Understanding RDMA helps when reading papers on disaggregated ANN, when evaluating specialized hardware for billion-scale graphs, or when comparing why a local in-memory HNSW hop is microseconds while a cross-rack fetch of the same neighbor list is orders slower without RDMA – and still non-trivial with it if the access pattern is chatty.

Choosing RDMA is therefore an infrastructure and software co-design decision.

When is RDMA worth considering for vector search?

Consider it when compute and memory pools are physically separated, when graph or posting fetches dominate latency, and when you can invest in contiguous layouts, batching, and pipelining so verbs carry large payloads. Skip it when shards are co-located with their vectors in ordinary servers – Weaviate’s default shared-nothing style – and when operational simplicity beats custom NICs and fabric tuning. Always compare end-to-end recall and p99 against a TCP baseline: RDMA that multiplies round-trips can lose to fewer, smarter RPCs. Pair any RDMA plan with the disaggregated-memory and network-versus-compute pages in this glossary; the hardware only helps if the ANN algorithm stops chasing pointers one cache line at a time across the wire.

RDMA is low-overhead remote memory access that can accelerate distributed ANN only when indexes are laid out for few large transfers. Next, read disaggregated memory for the architecture RDMA often serves, the network-versus-computation trade-off for when to move bytes versus recompute, and distributed graph traversal / query-aware data loading for how search schedules those remote fetches.