---
title: 'NCCL GIN: GDAKI Backend for GPU Networking'
url: https://www.emergentmind.com/topics/nccl-gin-gdaki-backend
type: topic
---

# NCCL GIN: GDAKI Backend for GPU Networking

NCCL GIN (GDAKI Backend) refers to the GPU-Initiated Networking (GIN) extension in NCCL 2.28, which enables fully device-driven, one-sided RDMA using the GPUDirect Async Kernel-Initiated (GDAKI) backend. This architecture eliminates CPU intervention in GPU-to-GPU network communication, permitting CUDA kernels to initiate, progress, and synchronize fine-grained transfers over InfiniBand or RoCE directly from the device. Developed to address the requirements of modern AI workloads such as Mixture-of-Experts (MoE), GIN GDAKI delivers low-latency and high-bandwidth communication that is tightly integrated with NCCL’s established collective operation infrastructure [2511.15076].

## 1. Three-Layer Architecture of NCCL GIN and GDAKI Placement

NCCL GIN is structured as a three-layer architecture:

- **NCCL Core (host-side):** Manages communicator creation (via `ncclDevCommCreate`), collective memory window registration (`ncclCommWindowRegister`), resource exchange, and GIN backend selection.
- **Device-GIN API (GPU-side):** Provides primitives—`put`, `putValue`, `signal`, local and remote completion counters/signals, `flush`, and `barrier`—callable directly from CUDA kernels.
- **GIN Network Plugin (backend):** Implements the active RDMA transport. There are two plugin modes:
  - **GDAKI:** Employs DOCA GPUNetIO to enable direct construction and posting of InfiniBand or RoCE work queue entries (WQEs), doorbell notification, and completion queue management, all from device code.
  - **Proxy:** Relays device requests as 64-byte descriptors via lock-free queues to a host thread, which posts verbs and updates completions.

GDAKI operates entirely within the network-plugin layer. If DOCA GPUNetIO is detected at communicator initialization, the GDAKI plugin (libnccl-net-gdaki.so) is loaded, ensuring all device API calls issue pure device-initiated communication with no CPU roundtrips.

## 2. Device-Side GDAKI API Semantics

GDAKI device API operations are strictly non-blocking, returning immediately to encourage computation/communication overlap. Completion is tracked via local counters or remote signals, all accessible from device memory. The key API elements include:

- Remote put: `put(team, peer, dstWin, dstOff, srcWin, srcOff, bytes, sig, ctr)`
- Zero-byte remote notification: `signal(team, peer, signalId)`
- Local completion management: counters and `flush`
- Remote completion management: signals and their polling/resetting
- Example (ring exchange in CUDA) demonstrates a peer-to-peer put followed by waiting on a remote signal and resetting it.

Semantically, a `put` generates a WQE in GPU-addressable BAR memory, followed by a doorbell write to the NIC’s register space. Completion is established either through local counter incrementation (modulo CQ) or remotely, when the corresponding signal counter is visible to the recipient GPU.

## 3. GDAKI Plugin internals and DOCA GPUNetIO

The GDAKI plugin, distributed as `libnccl-net-gdaki.so`, implements NCCL’s network API in "direct" device mode using DOCA GPUNetIO:

- **Resource Allocation:** Each context allocates specific NIC-side resources (queue pairs, completion queues) for the GPU via DOCA device verbs.
- **WQE Construction:** CUDA kernels format RDMA work requests into a BAR-mapped, GPU-accessible ring buffer.
- **Doorbell Mechanism:** Device code stores to the mapped NIC doorbell register to initiate DMA.
- **Completion Processing:** NIC hardware autonomously executes WQEs, conducts RDMA transfers, and writes CQEs into a completion queue mappable into GPU address space.
- **Signal/Counter Updates:** Local and remote completions are tracked by 64-bit counters (either on local or peer GPU BAR-mapped memory), which are polled or signaled by kernels.

In steady-state, this pipeline requires no active host thread involvement. Control flow, synchronization, and completion signaling are entirely managed by GPU code in conjunction with BAR-mapped device/NIC resources.

## 4. Performance Modeling and Empirical Results

RDMA latency and throughput under GDAKI are analytically and empirically characterized. Latency is modeled as:

$$ T_{\mathrm{RTT}}(n) = \alpha + \beta n $$

where $\alpha$ denotes round-trip doorbell/NIC/network fixed costs, and $\beta$ is the inverse bandwidth slope.

- **Microbenchmarks:** For 4–128 B ping-pong between two H100s, GDAKI yields $\alpha_\mathrm{GDAKI}\approx16.7\ \mu{\rm s}$.
- **Bandwidth:** Bulk transfer saturates the NIC’s port rate (e.g., 50 GB/s).
- **DeepEP MoE Kernels:** Throughput for $N_{\mathrm{ops}}$ put calls over total kernel execution time $T_{\mathrm{total}}$ is:
$$ BW = \frac{N_{\mathrm{bytes}} \times N_{\mathrm{ops}}}{T_{\mathrm{total}}} $$

### Benchmark Table

| Test Scenario                                          | NCCL GIN (GDAKI)         | NVSHMEM IBGDA      |
|--------------------------------------------------------|--------------------------|--------------------|
| 2-node dispatch BW (16 GPUs, BF16)                     | 84.36 GB/s               | 84.97 GB/s         |
| 8-node combine BW (64 GPUs, BF16)                      | 53.1 GB/s                | 54.0 GB/s          |
| Single-node dispatch latency (8 GPUs, 14 KB)           | 40.62 μs                 | 41.43 μs           |
| Two-node dispatch latency (14 KB)                      | 142.5 μs                 | 157.0 μs           |

All benchmarks were conducted on EOS DGX H100 clusters (ConnectX-7 InfiniBand, CUDA 12.x, NCCL 2.28, NVSHMEM 3.4.5).

## 5. Practical Integration Steps and Supported Workflows

For the GDAKI backend, system and integration requirements are:

- **Build:** CUDA 12.2+, ConnectX-6 Dx+ NICs, OFED/MOFED, and kernel modules (nv_peer_mem or dmabuf).
- **Runtime:** Set `NCCL_GIN_BACKEND=gdaki` to force direct device mode.
- **Communication Path Setup:**
  - Host-side: Create device communicator with `ncclDevCommCreate`, register memory windows collectively.
  - Device-side: In CUDA kernels, instantiate `ncclGin` objects and invoke `put`, `signal`, and `waitSignal` methods.
  - Cleanup: `ncclCommWindowUnregister`, `ncclDevCommDestroy`, and plugin teardown.

For MoE libraries such as DeepEP:
- Each expert-parallel communication channel maps to a dedicated GIN context (typically 4 per communicator).
- Kernels use `(window, offset)` pairs rather than raw pointers to identify buffers for RDMA.
- Circular buffer management is accomplished with `put` and `signalAdd`, coordinated through remote head/tail signals.

## 6. Constraints, Compatibilities, and Roadmap

- **Hardware/Software:** GDAKI requires NICs that export GPU-accessible doorbells and BAR-mapped WQE/CQ buffers (ConnectX-6 Dx or newer), CUDA 12.2+, and optimal results when NIC and GPU share a PCIe root complex.
- **Fallback:** The Proxy backend supports a broader range of NICs and GPUs (Volta+), sacrificing only modest extra latency.
- **Current limitations:**
  - Identical window size enforcement across ranks (NCCL 2.28).
  - No WQE batching; each `put` rings the NIC doorbell individually.
  - Static, bounded number of signal/counter IDs per communicator.
- **Planned enhancements:**
  - Batched doorbell notifications to reduce per-operation cost.
  - New one-sided primitives (e.g., remote atomics, TMA offload).
  - Context/QP multiplexing and adaptive NIC load balancing.
  - Fused collectives using device-primitives within NCCL’s algorithmic structures.

## 7. Significance and Areas of Application

GIN’s GDAKI backend brings zero-CPU-overhead, direct device-to-network offload to NCCL, achieving sub-17 μs single-message latencies and multi-tens of GB/s bandwidth. It is immediately applicable to MoE communication libraries (e.g., DeepEP), as well as to compiler-generated CUDA kernels that require native, inline network communication. The hardware/software design facilitates new computational paradigms tightly coupling computation and communication without host synchronization bottlenecks, while maintaining interoperability with NCCL’s production infrastructure [2511.15076].

Source: https://www.emergentmind.com/topics/nccl-gin-gdaki-backend