GNStor: Design of GPU-Native High-Performance Remote All-Flash Array

Abstract: GPU has become the leading computing device for a wide range of data-intensive applications, which tightly collaborates with remote all-flash array (AFA) to accommodate ever-expanding datasets, facilitate multi-client data sharing, and guarantee fault tolerance. Although GPU is the center of computation, all I/O processes in existing GPU-AFA systems are still CPU-centric. CPU orchestrates remote I/O requests and executes a centralized AFA engine to take charge of AFA-level functionalities (e.g., access control and metadata persistence). This design disparity suffers from substantial CPU-GPU interaction overhead and I/O traffic amplification, compromising end-to-end I/O performance. In this work, we present \emph{GNStor}, a GPU-native AFA system that enables GPU to directly access remote AFA without CPU intervention in the I/O path, thereby fully exploiting the performance of AFA. Specifically, GNStor first proposes a GPU-centric NVMe over RDMA (NoR) software stack (named \emph{GNoR}), paving a fast path for GPUs to directly initiate NoR I/O requests to SSDs within remote AFA. GNoR employs an atomic-operation-based I/O orchestration design and follows the single-instruction-multiple-thread (SIMT) execution model of GPU, fully exploiting the massive parallelism of GPU architectures. To facilitate essential AFA functionalities in a CPU-bypass I/O path, GNStor further designs \emph{deEngine}, a decentralized AFA engine that seamlessly decomposes and integrates AFA-level tasks into each SSD firmware, thereby achieving efficient AFA access at low cost. Evaluation results show that GNStor achieves 3.2$\times$ higher I/O throughput and reduces application execution time by 31.1\%, compared to state-of-the-art AFA systems.

• The paper presents a GPU-native design that bypasses CPU I/O, reducing latency by up to 39.8% and sustaining 3.2× higher IOPS.
• It integrates a GPU-centric NVMe-over-RDMA stack (GNoR) with a decentralized in-SSD engine (deEngine) for efficient metadata management and load balancing.
• Experimental evaluations show nearly linear scalability, 5–7× I/O speedup in key workloads, and a 31.1% reduction in end-to-end application time.

GNStor: GPU-Native High-Performance Remote All-Flash Array Architecture

Motivation and Problem Formulation

Emerging data-intensive workloads such as large-scale graph analytics, recommender systems, and LLM training/inference increasingly leverage GPU clusters connected to shared remote all-flash arrays (AFAs) for capacity, durability, and scalable data sharing. However, legacy GPU-AFA systems retain CPU-centric control planes and I/O orchestration, resulting in acute performance bottlenecks due to CPU-GPU synchronization, unnecessary I/O detours through host memory, amplified I/O traffic, and contention on centralized AFA engines. Despite evolving remote access, systems rely on the CPU for critical functions such as access control and metadata management, even when employing modern protocols like NVMe-over-RDMA (NoR) and hardware-accelerated target offload on NICs.

Figure 1: I/O path comparison of different GPU-AFA systems, highlighting CPU involvement in conventional systems versus direct GPU-to-AFA paths in GNStor.

Profiling reveals up to 79.1% throughput degradation and fundamentally limits the efficacy of high-throughput/low-latency GPU workloads with concurrent remote AFA accesses.

GNStor Architecture and Design Innovations

GNStor introduces a fundamentally GPU-native architecture, fully bypassing the CPU from the I/O datapath end-to-end, and efficiently decentralizing AFA functionality. The architecture consists of three primary components: GNoR (GPU-centric NVMe-over-RDMA software stack), deEngine (decentralized in-SSD AFA engine), and a lightweight CPU-resident GNStor daemon for infrequent management tasks.

Figure 3: GNStor system architecture, illustrating GNoR and deEngine roles in enabling GPU-native, highly parallel I/O and decentralized metadata control.

GNoR: GPU-centric Remote NVMe Stack

GNoR re-architects the NVMe-over-RDMA stack to operate natively from GPU threads. Unlike CPU-oriented NoR implementations, GNoR exploits the SIMT model and GPU parallelism by:

• Partitioning protocol-level responsibilities: Non-I/O-critical tasks (admin, keep-alive, connection setup) remain CPU-side; data-path I/O (command submission/completion, data movement) is mapped to GPU threads.
• Abstracting concurrency with GPU-resident “channels”: Each channel incorporates queue pairs, doorbell register access, pre-registered memory pools, and auxiliary metadata, eliminating contention and maximizing throughput.
• Replacing lock-based synchronization with atomic primitives for efficient thread coordination.
• Pre-allocating/registrating memory pools using a hierarchical, bitmap-based, fragmentation-minimizing GPU allocator matched to workload I/O sizes.

  Figure 2: GNoR channel initialization and I/O mechanisms: GPU-managed queues, direct NIC interaction, and atomic-based synchronization.

deEngine: SSD Firmware-Integrated Metadata and Access Control

GNStor decomposes and offloads AFA I/O-critical metadata and access control management from the centralized CPU engine into each SSD’s firmware (deEngine):

• Extends SSD firmware to maintain per-volume access control, persistently store mapping tables, and enforce consistency for many-to-many GPU-client/AFA-volume access.
• Coordinates replication and load balancing through hash-based volume addressing and decentralized, multi-versioned mapping metadata.
• Integrates mapping—volume/block address (VID/VBA) to SSD PPA—into the SSD FTL layer to avoid redundant lookups and consolidate durability/recovery mechanisms with native SSD power-loss protection.

  Figure 4: deEngine decentralized access control validation and metadata update logic on SSD controller.

  

  Figure 5: FTL table in deEngine: merged mapping from VID/VBA directly to SSD physical addresses.

Batched I/O, Native API Support, and System Integration

GNStor adapts to the SIMT execution nature of GPUs with explicit batched I/O mechanisms that minimize divergence and leverage warp-level primitives. It exposes an expressive CUDA library (libgnstor), providing both synchronous and async APIs, as well as explicit batched submission, completion, and memory management primitives.

Figure 6: Batched I/O coordination and completion scheme across GPU warps using shared memory bitmaps and atomic queues.

Figure 7: Subset of libgnstor APIs; direct GPU-side memory and I/O primitives facilitate integration into large-scale data pipelines.

Empirical Evaluation

Comprehensive microbenchmarks and application-level studies demonstrate the efficacy of GNStor.

• Microbenchmarks: GNStor sustains 3.2× higher IOPS versus CPU-centric designs on small I/O, and approaches the SSD/network hardware throughput limit in multi-client (32 clients, 4–5 SSD) configurations.
• Latency: Read and write latency are reduced by up to 39.8%, with consistent benefits across all I/O patterns.
• Scalability: Throughput scales linearly with both client and SSD count in the GPU-bypassed regime, strongly contrasting with scaling saturation in CPU-centric baselines.
• Ablation: deEngine accounts for up to 49.9% throughput gain via decentralized metadata, while GNoR achieves another ~49% gain by eliminating CPU orchestrator overhead.
• Application Case Studies: In representative tensor computing, data preprocessing, graph analytics, and LLM training workloads, GNStor reduces end-to-end application time by 31.1% (average), reaching 5–7× speedup in I/O-dominant tasks and maintains advantage as computation workload increases.

  Figure 8: Throughput comparison on microbenchmarks across baseline, GPUDirect, and GNStor.

  

  Figure 9: Latency comparison—GNStor consistently outperforms CPU-coordinated designs.

  

  Figure 10: Scalability test—GNStor nearly saturates SSD and network throughput as client number grows.

  

  Figure 11: Ablation—isolating deEngine and GNoR contributions to overall throughput gain.

  

Figure 14: Application-level benefits in tensor/matrix computing, data preprocessing, and graph analytics workloads.

Figure 12: LLM (GPT-2) training: GNStor delivers superior overall and checkpoint I/O performance, amplifying runtime benefits with model size.

Implications and Future Directions

The GNStor design paradigm constitutes a key rethinking of storage architecture for exascale GPU-centric workloads. By decentralizing management into the device layer and utilizing native GPU parallelism for protocol orchestration, GNStor eliminates critical system bottlenecks. This approach not only improves hardware utilization but also informs the co-design of future SSD firmware, network adaptors, and system integration for large-scale, shared GPU infrastructure. The architectural principles outlined here (in-protocol partitioning, firmware-anchored metadata, and atomic-synchronized GPU data path) can be directly extended to composable datacenter fabrics, in-SSD compute (computational storage), and privacy-preserving trusted execution environments.

The primary deployment barrier is SSD firmware extensibility—a challenge acknowledged, but increasingly addressable with the proliferation of open SSD platforms and computational storage devices. The GNStor software stack (GNoR) remains portable and generalizable to diverse GPU/NIC vendors subject to adherence to SIMT and NoR target offload capability.

Conclusion

GNStor presents a holistic, GPU-native, high-performance remote AFA system achieving maximal I/O path efficiency and scalability by entirely removing the CPU from the I/O critical path and decentralizing control logic into SSD firmware. Experimental evidence demonstrates substantial throughput and latency improvements alongside productive, scalable multi-client operation for data- and compute-intensive GPU workloads. This architecture provides a template for next-generation composable storage in GPU-based intelligent systems.

Reference: "GNStor: Design of GPU-Native High-Performance Remote All-Flash Array" (2606.04908)