---
title: NVIDIA NVLink-C2C Interconnect
url: https://www.emergentmind.com/topics/nvidia-nvlink-c2c-interconnect
type: topic
---

# NVIDIA NVLink-C2C Interconnect

NVIDIA NVLink-C2C Interconnect is a high-bandwidth, cache-coherent chip-to-chip (C2C) interconnect protocol and physical fabric, designed to drastically reduce host-device communication bottlenecks in heterogeneous high-performance computing (HPC), artificial intelligence (AI) inference, and large-scale data analytics workloads. Deployed across several GPU and “Superchip” product generations—including Tesla Volta, Ampere, Hopper, and Blackwell architectures—it acts as the foundational CPU↔GPU and GPU↔GPU transport, achieving bidirectional link bandwidths of up to 900 GB/s per pair and aggregate per-node figures exceeding 3 TB/s. NVLink-C2C’s evolution encompasses packet-based, mesh/ring topologies; full hardware cache coherence; protocol and lane aggregation; and software ecosystems that explicitly leverage its capabilities for MIG partitioning, serverless LLM inference, and mixed-precision out-of-core linear algebra.

## 1. Architectural Principles and Physical Layer Design

NVLink-C2C is architected as a direct, point-to-point or mesh-based bidirectional chip interconnect, contrasting PCIe’s shared bus. The core physical layer consists of multiple high-speed differential signaling lanes aggregated into logical “links.” In NVIDIA Tesla V100 (Volta), each of the six NVLink interfaces supports ≈25 GB/s per direction, yielding up to 300 GB/s aggregate full-duplex bandwidth per GPU when all links are enabled. Later platforms, notably the Grace Hopper (GH200) and Blackwell (H800) Superchips, employ fourth-generation NVLink-C2C, grouping up to four logical links per device at sustained per-link rates of 200–450 GB/s per direction. The protocol is cut-through, packet-switched, and lossless, utilizing flits (flow control units), CRC checks, and centralized per-link flow control. NVLink-C2C supports both DMA (Tensor Memory Accelerator, TMA) bulk transfers and cache-coherent, line-granular load/store and atomic operations across unified CPU–GPU memory spaces [1905.05175] [2410.09819] [2604.08451] [2510.15882].

C2C links sit below multi-instance GPU (MIG) slicing and are shared across all hardware partitions. MIG divides on-chip resources (HBM banks, SMs, L2 slices), but not the physical C2C links; thus, all slices contend on the same aggregate C2C fabric [2604.08451] [2605.19481]. 

## 2. Bandwidth, Latency, and Performance Modeling

Performance characteristics illustrate dramatic advances over PCIe. Link-level measurements on Volta and Ampere show unidirectional bandwidths up to 25–50 GB/s per link, totaling 300 GB/s per GPU with six links. On Grace Hopper and Blackwell, host–GPU C2C achieves 450 GB/s per direction (900 GB/s bidirectional), although practical throughput is constrained by host DRAM to ≈200 GB/s per direction for sustained large transfers. Latency is kept to single-digit microseconds (5–10 μs for 8 KB C2C transfers), a 2–5× reduction over PCIe Gen4/Gen5 [1905.05175] [2410.09819] [2601.20309].

A typical communication time model applies:

\[
T(\text{size}) = \alpha + \beta \cdot \text{size}
\]

where $\alpha$ denotes fixed launch/setup overhead, and $\beta \approx 1/B_{\text{link}}$ with $B_{\text{link}}$ the achieved link bandwidth. For bulk transfers, effective bandwidth exceeds 90% of theoretical when large, contiguous DMAs are used. Small transfers (<1 MB) see a lower effective share due to handshake overhead [1905.05175] [2604.08451] [2410.09819]. Highly optimized kernel and system design—e.g., pipelining, batching, overlapping compute and transfer across CUDA streams—enables sustained rates approaching 900 GB/s/node with four GH200 GPUs [2410.09819].

## 3. Topologies, NUMA Effects, and Multi-GPU Communication

NVLink-C2C topologies evolve across product generations:

- **Point-to-point Meshes:** V100/DGX-1 supports up to six links per GPU, forming partial meshes; direct peer-to-peer links achieve highest bandwidth/lowest latency. Multi-hop routing introduces NUMA effects, with latency penalties of ~1 μs per hop and a 20–30% remote bandwidth reduction [1903.04611].

- **NVSwitch:** In DGX-2 and supercomputer nodes (e.g., Summit, Alps, Leonardo), NVSwitch aggregates links into non-blocking, crossbar networks with per-pair unidirectional rates up to 1.2 Tb/s (H100) and node-wide aggregates exceeding 7 TB/s [2408.14090].

- **Superchip C2C (GH200/Blackwell):** These platforms couple one or more CPUs and GPUs with dedicated, full-coherence, chip-to-chip NVLink-C2C links. The interconnect presents a single, unified address space and enables direct CPU–GPU shared-memory semantics.

NUMA artifacts and path selection are critical in performance tuning. Direct connections—preferably dual/multi-link interfaces—should be used for latency- and bandwidth-critical paths; routing through intermediate GPUs or using single-link pairs introduces measurable penalties [1903.04611] [2408.14090].

| NVLink Generation | Max Unidirectional Bandwidth (per GPU pair) | Links per Pair | Typical Aggregate (per Node) |
|-------------------|--------------------------------------------|----------------|-----------------------------|
| V1 (P100)         | 800 Gb/s                                   | 4              | 4.8 Tb/s (4 GPUs)           |
| V2 (V100)         | 1.2 Tb/s                                   | 6              | 7.2 Tb/s (4 GPUs)           |
| 4.0 (H100/GH200)  | 1.2 Tb/s                                   | 6              | >7 Tb/s (4 GPUs)            |
| GH200 C2C         | 450 GB/s (per dir)                         | 2–4            | >1.8 TB/s (4 GPUs/CPUs)     |

All figures are per direction; bidirectional and aggregate totals are double [1903.04611] [2408.14090] [2410.09819].

## 4. Software Ecosystem: Partitioning, Offloading, and Scheduling

With the advent of MIG and high-bandwidth C2C, system software and runtime layers have developed to exploit NVLink-C2C’s bandwidth and coherence. In Hopper/GH200, C2C enables memory offloading: out-of-profile pages (e.g., Llama3 weights, FAISS indexes, Qiskit state vectors) are spilled from HBM to host DRAM, with SM-driven loads/stores fetching them on demand over C2C at rates up to 343 GB/s—even in the smallest (1g.12GB) MIG slices. Copy engine-based transfers (cudaMemcpy) are throttled by the static assignment of few copy engines per MIG partition, but kernel-based accesses fully saturate link capacity [2604.08451].

C2CServe interprets the C2C link as an active memory tier. By streaming large LLM weights from host DRAM to HBM on-demand with minimal cold-start overhead (e.g., <0.6 s for Llama-70B), it allows for elastic serverless LLM serving. Scheduling mechanisms balance HBM and C2C bandwidth via chunking and adaptive GEMM kernels (HybridGEMM) that trade-off streaming weights over C2C against on-chip reuse [2605.19481].

SuperInfer further demonstrates, under LLM batch serving pressure, SLO attainment rates for TTFT and TPOT improved by up to 74.7% under high C2C swap bandwidth (≈180 GB/s per direction) versus PCIe-bound baselines. Key optimizations include merging small KV cache entries into large DMA blocks (>4 MB), using batch transfer APIs to amortize launch overheads, and enabling cross-iteration pipelining to overlap transfers and compute [2601.20309].

## 5. Application Case Studies: Out-of-Core Linear Algebra and Collectives

In out-of-core tiled Cholesky on GH200, Reno et al. achieve ≈900 GB/s one-way host-attached GPU C2C performance, hiding virtually all communication beneath computation via static DAG-based scheduling. Utilization of mixed precision reduces tile transfer volume and further improves speedup (3× for 4-precision over FP64) while preserving accuracy [2410.09819].

Collective communication reveals both the strengths and current bottlenecks of NVLink-C2C. FlexLink, a load-balancing communication layer for H800 (Blackwell), coordinates traffic over NVLink, PCIe, and RDMA NIC paths, boosting AllReduce and AllGather bandwidth up to 27% by offloading selected traffic to secondary links when NVLink saturates. PCIe and RDMA contribute up to 22% of total collective traffic, with gains up to 26% in observed scenarios. This shows NVLink-C2C saturation remains possible for extreme-scale collectives, but multi-link software is capable of alleviating it [2510.15882].

## 6. Performance Bottlenecks, Scalability, and Optimization Strategies

Despite superior raw link rates, practical use of NVLink-C2C is sensitive to:

- **Transfer Size:** For sub-megabyte messages, launch/setup time dominates. Transfers batched into large, contiguous regions (≥4 MB) approach peak bandwidth.
- **Copy Path:** SM-driven in-kernel loads outperform traditional copy-engine driven cudaMemcpy in partitioned (MIG) environments.
- **C2C Contention:** Multi-tenant workloads, each streaming large footprints over the shared link, reduce per-slice effective bandwidth. Adaptive scheduling across partitions is needed to control interference [2605.19481] [2604.08451].
- **Power Isolation:** MIG partitions compute and memory, not power delivery, resulting in potential cross-slice throttling under compute saturation [2604.08451].
- **Software/Library Overheads:** Tuning NCCL/MPI chunk sizes, channel counts, and enabling GPU-aware DMA (GDRCopy, cudaMemcpyPeerAsync) are essential for sustaining performance, especially for collectives and small message regimes [1903.04611] [2408.14090].

Recommended strategies include aggregating small messages, maximizing data locality, exploiting asynchronous streams, and batching for DMA-style transfers. Library and OS-level optimization—such as NCCL parameter tuning, use of pinned memory, and CUDA peer strategies—allow practitioners to obtain >95% link utilization on well-structured workloads [1905.05175] [2408.14090].

## 7. Implications for Future Scalability and Software Systems

NVLink-C2C fundamentally alters the system design trade space for memory-bound and multi-tenant GPU-accelerated workloads. By moving large LLM weights, activations, or scientific data sets into CPU DRAM and leveraging hardware-coherent, near-HBM-rate access from the GPU, it enables elastic serving, fine-grained resource partitioning, and efficient out-of-core computation previously blocked by PCIe’s limitations [2605.19481] [2410.09819] [2604.08451].

Software systems must remain aware of contention, resource slicing, and NUMA path effects. Dynamic, feedback-driven scheduling (e.g., RotaSched, HybridGEMM, FlexLink) and allocation/admission control at the scheduler and kernel levels are critical for efficient resource sharing and to maximize SLO attainment in production environments [2601.20309] [2510.15882]. As aggregate and per-link rates increase, future research can further close the gap between observed and theoretical performance via active scheduling, cross-layer optimizations, and deeper integration with memory and virtualization ecosystems.

---

References: [1905.05175], [2604.08451], [2601.20309], [1903.04611], [2410.09819], [2605.19481], [2510.15882], [2408.14090]

Source: https://www.emergentmind.com/topics/nvidia-nvlink-c2c-interconnect