SpindleKV: SmartNIC Ordered KV Store

• SpindleKV is an ordered remote in-memory key-value store design that integrates SmartNIC data path accelerators and a learned index for efficient range operations.
• It leverages NIC-resident caching, stateless client protocols, and lock-free traversal to minimize PCIe crossings and reduce host memory access overhead.
• Its architecture addresses high memory latency challenges on SmartNICs while achieving impressive throughput for both point lookups and range queries under diverse workloads.

Searching arXiv for papers on SpindleKV and closely related SmartNIC ordered key-value stores. First I’ll search for “SpindleKV” directly, then for related terms like “ordered key-value SmartNIC learned index range queries”. SpindleKV is associated with the design space of ordered, range-capable remote in-memory key-value stores under SmartNIC constraints. In the literature relevant to SpindleKV, the BlueField-3-based system "Employ SmartNICs' Data Path Accelerators for Ordered Key-Value Stores" (Schimmelpfennig et al., 9 Jan 2026) is explicitly positioned as exploring the same design space: it builds an ordered KV store directly on the SmartNIC’s on-path Data Path Accelerators (DPAs), uses a lock-free learned index in accelerator-local memory, keeps clients stateless, and defers host access until the leaf level so as to minimize PCIe crossings. This positioning makes DPA-Store a concrete reference point for understanding SpindleKV’s likely architectural concerns, especially the joint requirement of peak efficiency, architectural simplicity, and native support for ordered operations.

1. Problem setting and system model

The relevant problem is the construction of a remote in-memory KV store that supports both high-speed point lookups and high-speed range scans without inheriting the major costs of conventional host-centric, hash-based, or RDMA-client-driven designs (Schimmelpfennig et al., 9 Jan 2026). Host-centric stores such as Redis, Memcached, or DPDK-bypassed systems still pay for the host networking stack and the PCIe boundary between NIC and CPU. Hash-based SmartNIC stores such as MICA or KV-Direct can deliver very high GET throughput, but they do not support range queries because hashing destroys order. RDMA-based ordered stores such as Sherman, XStore, and ROLEX do support ordered/range operations, but they often rely on stateful clients that cache metadata, maintain learned models, or otherwise participate in tree traversal and recovery. Honeycomb moves traversal onto the SmartNIC but suffers from repeated DMA round trips because it walks host memory on every level.

Within this design space, DPA-Store’s core response is to keep traversal logic and hot metadata on the NIC side, while only crossing PCIe when it absolutely has to fetch values or perform structural updates. This suggests that SpindleKV, insofar as it is aligned with the same design space, is best understood not as a generic SmartNIC KV cache, but as an architecture centered on ordered access, range functionality, and stateless clients rather than on pure hash-based GET acceleration.

A common misconception in this area is that SmartNIC acceleration naturally favors only unordered KV semantics. The DPA-Store result directly contradicts that assumption by combining SmartNIC-resident execution with a learned index that preserves order and enables range traversal.

2. DPA-centric architecture and hardware constraints

The motivation for using BlueField-3 DPAs is twofold: to avoid OS and host-network stack overheads, and to retain ordered operations without client state (Schimmelpfennig et al., 9 Jan 2026). DPA threads sit directly in the network datapath, so requests can be terminated on the SmartNIC without the kernel networking path. The cited system emphasizes that the DPA subsystem offers 16 physical RISC-V cores, 16 threads per core, for 256 hardware threads total, with 189 currently usable by applications. Requests can be steered directly to DPA threads via NIC packet matching rules.

The decisive architectural consideration is memory latency. The BlueField-3 DPA memory is reported at about 465 ns per access, and host DMA reads at around 910 ns. Because DPA memory access is much slower than a normal CPU DRAM access, the index design must minimize memory accesses. This is the reason the design adopts a learned tree: on the SmartNIC, reducing cache-line fetches and DMA operations matters more than it would on a CPU.

The resulting architecture places the learned index tree in NIC memory and keeps the high-capacity replica for actual values on the host. That arrangement preserves NIC-local traversal while avoiding exhaustion of the 1 GiB DPA memory. A plausible implication for SpindleKV is that its central challenge is not merely compute offload, but the asymmetry between abundant thread-level concurrency and high memory-access latency on current SmartNICs.

3. Learned index organization and ordered read path

The read path is completely NIC-driven until the leaf level and uses UDP over Ethernet, with one request per UDP packet (Schimmelpfennig et al., 9 Jan 2026). Packet matching rules on the BlueField-3 direct each listening UDP port to a designated DPA traverser thread. Every request can be served by every DPA thread, but port selection helps load balancing and cache locality. Clients default to a key-hash-based distribution across traverser threads.

The learned index tree itself lives in NIC memory. Inner nodes are split into seven learned-model segments; each segment stores a piecewise linear approximation (PLA) model, pivot keys, and child pointers. Each segment is limited to 128 pivot keys and child pointers. The traversal layout is tuned to the DPA cache hierarchy: the first cache line of an inner node contains metadata and the segment’s first keys, the second cache line contains the segment models, the next one or two cache lines contain pivots, and the final cache line contains the child pointer.

Traversal proceeds by binary search over segment keys, after which the chosen PLA model predicts a position

$p(k) = a \cdot k + b$

and the thread scans only a small window around it. The scan visits at most $2\varepsilon$ keys in $[p-\varepsilon, p+\varepsilon]$. The configured error bounds are $\varepsilon_{\text{inner}} = 4$ and $\varepsilon_{\text{leaf}} = 8$, yielding at most two cache-line accesses for inner-node search and at most three DMA-accessed cache lines at the leaf. Because BlueField-3 DPAs do not support floating-point operations, the implementation uses fixed-point arithmetic and temporarily expands 64-bit keys to 128-bit for precise calculations.

For a GET, once the traverser reaches the leaf, it scans the leaf’s insert buffer first, scans the host-resident leaf key array around the predicted position if necessary, and then DMA-reads the value from host memory. The leaf keys array resides in host memory and is prefetched during model computation to hide some of the DMA latency. For a RANGE query, the traverser descends to the leaf for the smallest key $k_{\min}$, scans the insert buffer first, scans the host-resident leaf arrays, and, if necessary, continues by descending to the next leaf using the smallest key strictly larger than the last returned key.

These details are central to the ordered semantics relevant to SpindleKV. Ordered and range access are supported because the learned tree preserves key order and leaves are stored in sorted layout. The host replica is consulted only when the traverser is already at the correct leaf region, which keeps PCIe crossings minimal.

4. Update staging, batching, and consistency semantics

The write path is split between NIC-side staging and host-side structural maintenance (Schimmelpfennig et al., 9 Jan 2026). INSERT, UPDATE, and DELETE requests do not immediately mutate the tree structure. Instead, at the leaf level they are appended to a per-leaf insert buffer on the NIC. Each buffer write uses two atomic counters: one increment before writing the record, and one increment after writing the record. Lookups read values before the corresponding key to preserve correct mapping. Once appended, updates become immediately visible to later lookups because the insert buffer is checked first on every read. If the insert buffer is full, the traverser re-enqueues the request for later processing.

When the last slot in an insert buffer is filled, the traverser sends a patch request to the host via DMA. Only the traverser that filled the buffer can emit this request, and only one patch request can be triggered per buffer. On the host, patcher threads process patch requests. The host applies UPDATEs directly to values, merges INSERT/DELETEs with existing leaf contents, retrains affected PLA segments, and splits or rebuilds nodes as needed. For INSERT/DELETE patches, the host merges the updates with the leaf’s current keys into a temporary sorted array and partitions that array using PLA segmentation with the configured $\varepsilon_{\text{leaf}}$. If necessary, the host rebuilds parent levels bottom-up using the same PLA approach with $\varepsilon_{\text{inner}}$. When splitting becomes necessary, segment sizes are limited by a retrain bound of 0.25 × capacity, intentionally leaving slack so future patches can be absorbed without another split.

Once the host has built the new tree fragments, it returns them to the NIC as COPY and CONNECT stitch commands. COPY stitches place new nodes in DPA memory; a CONNECT stitch performs a pointer swap to make the new subtree visible. The update is applied using RCU semantics: the tree is never modified in place, and the old version remains traversable until the new version is fully stitched in. The tree is partitioned beneath the first inner level so multiple stitcher threads can work concurrently on disjoint partitions. For root-level updates, the system uses UIDs to check whether a node is already present before a CONNECT is applied, and a queue fence blocks the root-install stitch until earlier updates are complete. Obsolete nodes are reclaimed later using epoch-based reclamation, derived from packet counters.

The significance for SpindleKV is that ordered, lock-free traversal is achieved not by making traversers update structure in place, but by separating fast-path staging from host-side structural rebuild and transactional stitching.

5. Stateless clients, NIC-resident caching, and concurrency control

The client side is intentionally stateless (Schimmelpfennig et al., 9 Jan 2026). Clients send UDP packets, optionally use key hashing to distribute load across DPA threads, optionally include cache-lookup metadata, and retransmit if a response is not received in time. They do not maintain a persistent index replica, cache subtree locations, or manage consistency metadata. UDP is used rather than TCP because the DPA request path is event-based and not suited to a full TCP stack with sequence tracking and retransmission logic.

The NIC-side read cache is per traverser thread and consists of a three-way Bloom filter, a hash table of cache-line-sized buckets, and four KV pairs per bucket. The client sends extra lookup metadata—Bloom hash index and hash bucket index—so the DPA can avoid recomputing those values. The Bloom filter is 256 bits, chosen because it fits into the remaining space of the traverser thread’s context cache line; the paper stresses that this cache line is “nearly never evicted” from the DPA thread’s L1, so the Bloom filter lookup is effectively free. The cache stores both keys and values to detect hash collisions correctly. UPDATE and DELETE invalidate corresponding cache entries. The design deliberately avoids active hotness tracking because it would cost additional counters and memory accesses.

The cache configuration is 96 entries per traverser thread, 176 traverser threads, and 16,896 total cached entries, with an average false-positive rate of 31%. On a Zipfian workload with $\alpha = 1$ and a 200M-entry dataset, the cached entries cover more than 50% of requests. If values are chosen randomly for caching, the overall hit ratio is still about 25%. Under skew, the cache increases throughput by up to 30%.

Concurrency control is explicitly lock-free on the traversal side. Traversers never mutate the tree in place. Atomic counters protect insert-buffer appends; multiple DPA threads serve requests concurrently because requests are hashed to threads; the host uses exclusive parent locking during structural rebuilds; cache consistency is maintained by invalidating UPDATE/DELETE entries and by storing both key and value in the cache; and epoch-based reclamation prevents freeing nodes that a traverser might still reach through an outdated path. A frequent misunderstanding is that stateless clients imply weak consistency or trivial request processing. In this design, statelessness is paired with substantial NIC-side request intelligence, cache handling, and lock-free traversal semantics.

6. Evaluation, comparative results, and limitations

The cited implementation reports 33 MOPS for point lookups and 13 MOPS for range queries (Schimmelpfennig et al., 9 Jan 2026). It also reports low insert throughput, only 1.7 MOPS for INSERT-heavy workloads, and UPDATE-only workloads reaching up to 12.1 MOPS. The experimental setup uses one BlueField-3 B3140L server, plus six client machines, a 100 Gb/s Ethernet switch supporting RoCE, Mellanox ConnectX-5 client NICs with DPDK, and a BlueField-3 in NIC mode with DPA DMA access to host memory and disabled ARM cores. Datasets include SOSD benchmarks—sparse, dense4x, face, amzn, wiki, osmc—with 25M entries bulk loaded before each experiment unless otherwise stated; Zipf skew typically uses $\alpha = 0.99$; and both values and keys are 64-bit. Throughput is averaged over four runs for long benchmarks and eight runs for short ones; client start times are synchronized via MPI; standard deviation is below 5% for throughput and below 9% for latency.

Sensitivity analysis shows that throughput scales roughly with traverser thread count. Above 16 traverser threads, multiple DPA hardware cores are used and latency improves. The chosen operating point is 176 traverser threads on 11 DPA cores, plus 4 stitcher threads on a dedicated core, because mixing traversers and stitchers on one core hurts INSERT throughput by about 14% due to NIC doorbell event prioritization. Host-side patchers are set to 4 threads. Client queue depth is 32 for GET and 18 for insert/range workloads. Deep trees slightly reduce GET throughput because of the extra inner-node access. Larger $2\varepsilon$0 values reduce memory overhead but increase search cost; datasets like face and osmc need larger $2\varepsilon$1 to avoid excessive memory usage.

The analytical model for traversal time, for depth 3 and uncached accesses, is

$2\varepsilon$2

with

$2\varepsilon$3

which implies

$2\varepsilon$4

If the root’s first cache lines are cached and replaced with 64 ns L3 accesses, the computed upper bound is 31.05 MOPS. The authors further note that with 100 ns memory latencies, GET latency could fall below 2.82 µs and throughput rise above 62 MOPS.

Bulk loading 50M sparse entries took 1,643 ms total, with 1,605 ms spent copying and processing stitch requests. The effective host-to-DPA transfer bandwidth was only 120 MB/s for 192 MB of tree data. INSERT-heavy workloads suffer from the same low host-to-DPA bandwidth, which is identified as the major bottleneck for the 1.7 MOPS insert ceiling. The paper states explicitly that future SmartNICs need a better interface for efficiently transferring data from host memory to DPA memory.

The comparison with ROLEX is particularly relevant to SpindleKV’s design space. DPA-Store exceeds ROLEX throughput on amzn and osmc for YCSB-A and for all range-only workloads. It also does better on GET-only workloads for sparse and amzn, with lower latency. ROLEX does better on osmc because its larger $2\varepsilon$5 is more suitable for that dataset, and it still wins on INSERT throughput because its clients directly RDMA inserts into server memory and can decouple model retraining more lazily. DPA-Store nearly matches ROLEX in mixed read-heavy workloads with only a small insert fraction, and has lower latency in all experiments because ROLEX pays for RDMA round trips and client-side state, while DPA-Store can satisfy many accesses from the NIC-side cache and insert buffer.

The stated limitations are explicit: DPA memory latency is the main GET bottleneck; host-to-DPA writes are too slow; the architecture is compute-rich but memory-latency constrained; packet steering skew can overload individual DPA threads; and UDP implies no built-in reliability beyond client-side timeout and resend. For SpindleKV, the main implication is that future gains are likely to depend more on improving SmartNIC memory access and host-to-NIC data movement than on changing the high-level algorithmic structure alone.