NCCLbpf: eBPF Framework for Safe GPU Collectives

• NCCLbpf is a high-performance extension that embeds a userspace eBPF runtime into existing NCCL plugin interfaces, ensuring safety by statically verifying policies at load time.
• It employs structured, lock-free eBPF maps and atomic hot-reload to enable closed-loop adaptation and seamless policy updates without interrupting multi-GPU jobs.
• Through zero-change integration with NCCL, NCCLbpf achieves significant throughput improvements in specific message-size ranges while mitigating risks of crashes, hangs, and memory corruption.

NCCLbpf is a verified, high-performance extension framework for GPU collective communication that embeds a userspace eBPF runtime into NCCL’s existing plugin interfaces, without modifying NCCL itself. It is designed to address a specific safety and operability problem in NCCL’s plugin model: NCCL plugins execute as unverified native code within NCCL’s address space, so a single defect can crash or hang a multi-GPU job, silently corrupt internal state, or force downtime when policies are updated. NCCLbpf introduces load-time static verification, structured cross-plugin maps, and atomic policy hot-reloads, while preserving NCCL’s existing ABI and the hot-path performance characteristics required by large-scale distributed training (Zheng, 12 Mar 2026).

1. Position within NCCL’s plugin model

Large-scale distributed GPU training frameworks, including Megatron-LM, DeepSpeed, and PyTorch FSDP, rely on NCCL for inter-GPU collectives such as AllReduce, AllGather, and Broadcast. NCCL’s plugin model exposes four loadable shared libraries—tuner, profiler, net, and env—each injected via dlopen into NCCL’s process space (Zheng, 12 Mar 2026).

The plugin roles are differentiated at the ABI level. The tuner plugin (getCollInfo) picks algorithm, protocol, and channel count by adjusting cost tables; the profiler plugin receives timestamped tracepoints; the net plugin controls network transport; and the env plugin sets runtime parameters. In NCCL’s default model, these components are extensibility points, but they remain ordinary native shared libraries executing in-process.

The significance of NCCLbpf lies in reusing these existing plugin interfaces rather than replacing NCCL’s architecture. This preserves deployment compatibility while changing the trust model around policy execution. A plausible implication is that NCCLbpf targets operationally sensitive environments where maintaining NCCL compatibility is as important as improving policy flexibility.

2. Safety gap and design objectives

The motivating problem is that NCCL plugins are unverified native code. The paper identifies null-pointer dereference, infinite loop, race condition, and subtle memory corruption as representative failure modes. Because the code runs inside NCCL’s address space, a single bug can crash or hang an entire multi-GPU job or corrupt internal NCCL state; moreover, the lack of hot-reload means any policy update forces a disruptive job restart (Zheng, 12 Mar 2026).

NCCLbpf closes this “safety gap” by importing three properties associated with eBPF-based extensibility. First, load-time static verification performs memory-safety checks, bounded-loop checks, and stack checks before a policy is ever invoked. Second, typed eBPF maps provide structured, lock-free state sharing across policies, enabling profiler-to-tuner feedback and other closed-loop adaptations. Third, atomic program replacement enables policy hot-reload without dropping collective calls.

These properties change both failure handling and policy lifecycle management. In the native model, policy changes and policy faults are runtime risks. In NCCLbpf, unsafe programs are rejected before execution, and policy replacement is performed atomically. This suggests that NCCLbpf treats plugin logic less as a privileged library extension and more as a verified, dynamically replaceable policy object.

3. Architecture and execution model

NCCLbpf’s integration is described as zero-change: a single shared library registers as both an ncclTunerPlugin (v3/v5) and ncclProfilerPlugin (v1/v6). Inside that host library, a bpftime userspace runtime provides a PREVAIL-based verifier, an LLVM JIT compiler, a map subsystem, and helper functions. Policy authors compile restricted C to BPF ELF objects, which are then loaded through NCCLbpf’s host-side machinery (Zheng, 12 Mar 2026).

At plugin load time, the host library performs four steps: it loads the ELF, runs the PREVAIL verifier, JIT-compiles each SEC("tuner") or SEC("profiler") function, and resolves symbols such as maps and helper stubs. The paper reports that this verification step takes 1–5 ms one time. Because all of this happens before any collective call, no unverified code executes in the hot path.

Cross-plugin composability is realized through typed eBPF maps. The paper’s example is a latency_map keyed by communicator ID and holding per-communicator performance statistics. The profiler program writes observed latencies, and the tuner program later reads them to inform adaptive channel selection. This provides a structured mechanism for state exchange across policies, in contrast to ad hoc shared-memory coordination.

Atomic hot-reload is implemented by storing the active policy function pointer in an atomic variable. Updating a policy consists of loading a new BPF ELF, verifying and JIT-compiling it to x86_64 machine code, and then atomically CAS-swapping the old function pointer for the new one. In-flight calls finish under the old pointer, and the next call uses the new policy. If verification fails, the swap is aborted and the old policy continues.

4. Verification, maps, and composability semantics

The bpftime runtime comprises four principal subsystems: a verifier, a loader, a JIT, and maps. The verifier performs PREVAIL-based static checks for memory safety, bounded loops, stack depth, and helper whitelisting. The loader parses BPF ELF sections and sets up map metadata and helper trampolines. The JIT uses LLVM-based compilation to optimized x86_64. Maps are typed array or hash-map structures with atomic lookup, update, and delete (Zheng, 12 Mar 2026).

A notable implementation detail concerns communicator identification. Because NCCL’s tuner API lacks a direct integer ID, NCCLbpf hashes the ncclComm_t* pointer to a 32-bit key for map lookup. This is a concrete example of adapting eBPF-style map semantics to an existing NCCL ABI rather than requiring ABI changes.

The verifier enforces specific safety rules. It rejects any BPF program that dereferences a map lookup result without a preceding null-check, writes to input fields, performs unbounded loops, or invokes illegal helpers. The paper also states that structured maps enforce sharing only fixed-size key/value pairs, which are described as ideal for per-communicator scalars while eliminating ad hoc shared-memory bugs.

The composability mechanism is central to NCCLbpf’s design. The profiler-to-tuner path via shared maps enables closed-loop adaptation that the paper describes as impossible in NCCL’s native plugin model. The key point is not merely that state can be shared, but that it is shared through a verified, typed, and constrained abstraction.

5. Performance characteristics and empirical results

The evaluation uses 8× NVIDIA B300 SXM6 GPUs (Blackwell, 275 GiB), NVLink5 interconnect (1.8 TB/s per GPU), CUDA 13.0, and NCCL 2.29.7. CPU microbenchmarks run on a 240-core AMD EPYC 9575F. Overhead measurements are reported over 1 million calls for various eBPF policies versus a native C++ baseline, with all code compiled using -O2 (Zheng, 12 Mar 2026).

The paper reports that the “noop” eBPF policy pays an extra 80 ns P50 overhead, decomposed into 80 ns framework base, of which 33 ns is JIT dispatch, plus 30 ns per map lookup and 10 ns per map update. The most complex policy, slo_enforcer, incurs +130 ns P50. At P99, overhead is 111–160 ns. For a 128 MiB 8-GPU AllReduce with latency approximately 394 μs, the worst-case 130 ns policy decision corresponds to about 0.033%.

A message-size sweep from 4 MiB to 8 GiB shows that NCCL’s default NVLS algorithm is suboptimal in the 4–128 MiB range. The evaluated eBPF policy, nvlink_ring_mid_v2, applies the following rules: for 4–32 MiB it selects Ring/LL128 with 32 channels; for 64–192 MiB it selects Ring/Simple with 32 channels; otherwise it defers to the NCCL default.

The throughput results reported for 8-GPU AllReduce are as follows:

Size	Default (NVLS)	eBPF Policy
4 MiB	133.5 GB/s	148.1 GB/s
8 MiB	196.3 GB/s	249.7 GB/s
16 MiB	278.8 GB/s	337.4 GB/s
32 MiB	349.3 GB/s	402.4 GB/s
64 MiB	425.2 GB/s	471.8 GB/s
128 MiB	596.9 GB/s	628.9 GB/s

The corresponding reported improvements are +10.9%, +27.2%, +21.0%, +15.2%, +11.0%, and +5.4%, with the best gain at 8 MiB. For sizes above 256 MiB, the default NVLS policy wins and the eBPF policy falls back accordingly. The paper summarizes this as an AllReduce throughput improvement of up to 27% over NCCL’s default in the 4–128 MiB range.

6. Hot-reload behavior, safety evaluation, and use cases

The safety evaluation tested 14 BPF programs: 7 safe and 7 unsafe. All safe programs were accepted, and all unsafe variants were rejected at load time. The unsafe set included null-deref, OOB access, illegal helper, stack overflow, infinite loop, input-field write, and division by zero. The comparison given in the paper is explicit: a native plugin with a null-deref crashes immediately with SIGSEGV, whereas the eBPF version is rejected with the message VERIFIER REJECT: must check != NULL before dereference at insn 7 (Zheng, 12 Mar 2026).

For hot-reload, the paper reports ~9.4 ms end-to-end replacement time, of which only 1.07 ms is on the hot path. Across 400 000 continuous calls during policy hot-reload, zero lost calls were observed. The update protocol therefore provides both admission control and continuity: unverified code never executes, and verification failure leaves the prior policy active.

The composability case study uses an adaptive channel policy in which the profiler writes average latency to latency_map, and the tuner reads it to adjust channel count according to a threshold at 1 ms. In 100 000 calls, channels ramp from 2 to 12 under low load, drop back to 2 under injected contention, and recover to 12 upon resolution. The paper states that this coordination is impossible in NCCL’s native plugin model.

The stated use cases fall into three categories. Under safety, NCCLbpf prevents crashes and silent memory corruption by catching bugs at load time and eliminates hangs from infinite loops or races. Under operability, atomic hot-reload avoids job restarts during policy updates and structured maps replace ad hoc shared state, removing locking bugs. Under policy expressiveness, the paper highlights message-size-aware tuning in fewer than 20 lines of C, closed-loop adaptation via profiler-to-tuner maps, and net plugin instrumentation of bytes and connection counts with less than 2% overhead.

7. Interpretation and relation to GPU communication policy design

NCCLbpf is best understood as a re-specification of NCCL plugin execution semantics rather than merely a faster or safer plugin implementation. The framework leaves NCCL itself unchanged, but it alters when policy code is admitted, how policy code communicates, and how policy code is replaced. Verification is moved to load time, state exchange is mediated by typed maps, and updates become atomic pointer swaps instead of restart events (Zheng, 12 Mar 2026).

Its main contribution is therefore compositional. A tuner policy, a profiler policy, and shared maps are not separate conveniences; together they form a policy substrate for closed-loop control over collective communication. The paper’s adaptive channel example illustrates this explicitly, and the throughput case study shows that such policies can also outperform NCCL’s default selection in specific message-size regimes.

Common misconceptions can arise from conflating NCCLbpf with a modification of NCCL internals or with a generic plugin sandbox. The paper does not describe either. It describes a single shared library that registers through NCCL’s existing plugin ABI and embeds a userspace eBPF runtime. Likewise, the framework is not presented as a general proof of optimal collective selection; its concrete performance result is tied to a message-size-aware eBPF policy and to the observed suboptimality of default NVLS in the 4–128 MiB range.

The broader implication is that GPU collective policy can be made simultaneously safer, more composable, and operationally easier to update, without sacrificing hot-path efficiency. This suggests a model in which collective-communication policy becomes a verified runtime artifact rather than an in-process native extension, while remaining deployable within today’s NCCL-based training stacks.