NVIDIA H200: Confidential Hopper GPU

• NVIDIA H200 is a Hopper-series GPU featuring a 700W power cap, 144 GiB of HBM3e memory, and 4.89 TB/s bandwidth for high-throughput and memory-centric workloads.
• It integrates robust confidential computing with hardware-backed attestation, secure CPU–GPU communication, and AES-256-GCM encryption to protect sensitive AI processes.
• The H200 demonstrates workload-dependent performance trade-offs, excelling in memory-bound tasks like LLM inference and spectral Ewald summation while serving as a key baseline for post-Hopper designs.

Searching arXiv for recent H200-related papers to ground the article. arxiv_search(query="NVIDIA H200 confidential GPU H200 HBM3e power capping Hopper H200", max_results=10) arxiv_search(query="(Schambach et al., 30 Jun 2026) OR (Ujeniya et al., 13 Apr 2026) OR (Kosmacher et al., 17 Jun 2026) NVIDIA H200", max_results=10) arxiv_search(query="EnclaveX End-to-End Confidential AI with CPU/GPU TEEs (Schambach et al., 30 Jun 2026)", max_results=5) NVIDIA H200 is presented in recent research as a Hopper-series GPU whose defining published characteristics include a 700 W power draw limit, 144 GiB memory capacity, HBM3e memory technology, 3201 MHz memory frequency, 4.89 TB/s memory bandwidth, 1665 MHz base SM frequency, 1980 MHz boost SM frequency, and 67 TFlop/s TF64 peak throughput (Ujeniya et al., 13 Apr 2026). Across papers, it appears in three especially important roles: as a confidential GPU (cGPU) in unified CPU/GPU trusted-execution workflows, as a memory-bandwidth-centric Hopper device whose HBM3e subsystem changes power-efficiency trade-offs relative to H100, and as a practical accelerator for large-scale AI and scientific kernels, including confidential LLM inference and spectral Ewald summation (Schambach et al., 30 Jun 2026, Kosmacher et al., 17 Jun 2026).

1. Architectural profile within the Hopper generation

The published comparative characterization of H200 emphasizes that its compute characteristics are deliberately close to H100, while its memory subsystem is substantially different. In the comparative study of power capping, H100 and H200 share the same 700 W limit, the same 1665 MHz base SM frequency, the same 1980 MHz boost SM frequency, and the same 67 TFlop/s TF64 peak; the major architectural difference is memory, with H100 using HBM2e at 1593 MHz and H200 using HBM3e at 3201 MHz, alongside a bandwidth change from 2.41 TB/s to 4.89 TB/s and capacity change from 94 GiB to 144 GiB (Ujeniya et al., 13 Apr 2026).

Metric	H100	H200
Power draw limit	700 W	700 W
Memory capacity	94 GiB	144 GiB
Memory technology	HBM2e	HBM3e
Memory frequency	1593 MHz	3201 MHz
Memory bandwidth	2.41 TB/s	4.89 TB/s
Base SM frequency	1665 MHz	1665 MHz
Boost SM frequency	1980 MHz	1980 MHz
TF64 peak	67 TFlop/s	67 TFlop/s

This profile matters because the literature does not frame H200 as a compute-throughput redesign in the narrow SM-count sense. Instead, the core architectural claim is that H200 “does not gain compute throughput from more SMs or higher SM peak; it gains from faster memory, which indirectly changes power allocation and DVFS behavior” (Ujeniya et al., 13 Apr 2026). That framing recurs across application papers: when H200 is advantageous, the advantage usually arises from bandwidth, capacity, or deployment mode rather than from a wholly new execution model.

2. Confidential GPU semantics and attestation

A major 2026 line of work positions H200 as a confidential GPU inside end-to-end confidential AI systems. In that setting, H200 is not treated merely as a faster accelerator, but as the GPU component of a TEE-backed workflow in which a confidential VM on the CPU side and the H200 on the GPU side jointly protect LLM inference, training, and fine-tuning (Schambach et al., 30 Jun 2026).

The H200/Hopper platform is described there as supporting “a unified TEE that spans both CPU and GPU,” with confidentiality covering GPU memory and registers, CPU–GPU communications, and the I/O path over PCIe. The protection mechanism is hardware-backed: both sides protect I/O over otherwise untrusted PCIe links using bounce buffers backed by hardware AES-256-GCM, and the TEEs negotiate a shared key through SPDM. In the evaluated configuration, the H200 is attached to an Intel TDX confidential VM; during boot, the VM attests its firmware, guest OS, and custom kernel module, that module requests the GPU driver to attest the confidential GPU, and only after successful attestation does the control plane release signing keys and secrets (Schambach et al., 30 Jun 2026).

The GPU-side attestation model is more detailed than a generic “device present” check. The relying party challenges the GPU with a nonce; the H200 produces an attestation report signed by a GPU-generated Attestation Key (AK); the verifier checks a certificate chain including NVIDIA GPU Driver certificates and revocation status via NVIDIA OCSP; and the report is matched against NVIDIA Reference Integrity Manifest (RIM) files to derive golden measurements. The paper’s stated security condition is explicit: “If all report measurements match with a golden measurement, the GPU is assumed to be in an expected state” (Schambach et al., 30 Jun 2026).

For AI deployment, the significance is that H200 is presented as enabling protected accelerator execution rather than only encrypted host memory. The accelerator-specific guarantees named in the paper include GPU memory and register confidentiality, secure CPU–GPU communication, attestation that the GPU is a genuine NVIDIA Hopper GPU in CC-enabled mode, measurement-based verification of driver and firmware state, nonce-based freshness, AES-256-GCM encryption over bounce buffers, and SPDM-based key negotiation (Schambach et al., 30 Jun 2026).

3. Trust boundaries, cloud orchestration, and residual risk

The confidential-computing literature around H200 is explicit that GPU TEE support does not by itself collapse the cloud trust boundary. One of the central deployment findings is that in Kubernetes environments, administrators remain highly privileged: a K8s admin can still run kubectl exec into a confidential VM and access sensitive data, “effectively operating as a root user inside the secure enclave” (Schambach et al., 30 Jun 2026).

The mitigation strategy in the same work is layered rather than purely hardware-centric. Secrets are released only after successful attestation, so a K8s administrator who can enter the VM still cannot access encryption keys; the system additionally disables memory dump features in the Guest kernel to prevent dumping the application’s memory inside the CVM. The threat model also assumes a Dolev–Yao-style network adversary able to drop, inject, replay, or modify packets, while treating side-channel attacks and denial of service as out of scope. The attestation stack itself is described as somewhat pre-production: in the benchmark setup, Intel quote verification could not fully validate pre-production quotes through QvL, and the verifier did not fully support all H200 report measurements, so unknown measurements were skipped and the rest verified (Schambach et al., 30 Jun 2026).

These constraints correct a common misconception. H200 confidential mode adds strong accelerator-side confidentiality, but the paper’s operational message is that secure deployment still depends on the surrounding CVM + driver + attestation chain, on application-level secret release, and on hardening against the management-plane threat represented by Kubernetes administration. The confidentiality properties are therefore substantial, but they are conditional rather than absolute (Schambach et al., 30 Jun 2026).

4. Power capping, HBM3e, and energy-efficiency trade-offs

The clearest quantitative study of H200’s architectural trade-offs isolates the HBM3e subsystem as the key variable. The paper models total board power as

$$W_{\mathrm{Total}} = W_{\mathrm{GPU}} + W_{\mathrm{Memory}},$$

where $W_{\mathrm{GPU}}$ covers SMs, L2, and the memory controller, and $W_{\mathrm{Memory}}$ covers the HBM subsystem (Ujeniya et al., 13 Apr 2026).

Under power caps from 200 W to 700 W in 100 W increments, the H200’s memory system draws substantially more power than H100’s. The reported maximum memory power draw is about 120 W for H100 and about 220 W for H200, with outliers around 240 W on some H200 specimens; the paper also reports saturation behavior around 125 W for H100 memory power and around 220 W for H200 memory power (Ujeniya et al., 13 Apr 2026). The qualitative consequence is an inverse relationship between memory power and available SM frequency: H200’s faster HBM3e consumes more of the board budget, leaving less headroom for the SMs.

That trade-off produces a workload-dependent hierarchy rather than a universally better device. For DGEMM, chosen as the compute-bound Roofline extreme, H100 slightly outperforms H200 at the same power cap, and the study concludes that H100 is the slightly better choice for strictly compute-bound workloads. The stated reason is straightforward: H200’s memory subsystem draws more power, so SM frequency is throttled more aggressively in a kernel whose performance depends strongly on SM frequency (Ujeniya et al., 13 Apr 2026).

For the Schönauer Triad memory-bandwidth benchmark, the result reverses. H100 reaches peak effective memory bandwidth at about 350 W, whereas H200 needs at least about 550 W to sustain its larger memory bandwidth; nonetheless, H200 is clearly more efficient than H100 across all power caps in this memory-bound regime, and the paper highlights that H200 at 400 W exceeds H100 at full TDP in efficiency (Ujeniya et al., 13 Apr 2026). The benefit of H200 is therefore tied to the ability of the workload to exploit its added bandwidth.

The same study also reports a low-cap anomaly that is important for operational planning: at a 200 W cap under memory-bound load, H200 does not actually remain at 200 W, with observed average power around 250 W. The interpretation given in the paper is that the memory subsystem’s baseline power demand is high enough that the GPU cannot fully honor a strict low cap in that regime, even while SM frequency collapses to the base frequency of 345 MHz (Ujeniya et al., 13 Apr 2026).

5. Software optimization, partitioning, and Hopper-specific portability

Two other papers situate H200 within Hopper-era software and systems techniques, but both do so partly by extrapolation from other devices rather than by direct H200 measurement. One compiler paper evaluates only H100, yet its central hardware target is Hopper’s Distributed Shared Memory (DSM) and thread-block cluster mechanism rather than a uniquely H100-specific feature set. The authors characterize DSM as an inter-core connection that links the shared memory of multiple SMs in a cluster into a larger on-chip memory pool, and the paper argues that this can expand fusion beyond single-SM register/SMEM limits for operator chains such as FFNs (Huang et al., 15 Dec 2025).

Because that work does not benchmark H200, any H200 conclusion is inferential. A plausible implication is that DSM-aware fusion strategies remain relevant on H200 so long as H200 preserves Hopper’s thread-block cluster and DSM semantics. The paper itself frames the mechanism as “Hopper DSM-specific,” and explicitly states that the strongest transferable expectation is that the same compiler strategy—keeping intermediates on-chip through DSM-backed inter-core communication—should carry over, while exact speedups would require remeasurement on H200 (Huang et al., 15 Dec 2025).

A separate systems paper treats H200 as a Hopper-series MIG-capable GPU that offers both performance and security isolation features, but again without direct H200 experiments. Its measured platform is A100 40GB PCIe, while H200 enters as part of the architectural scope of modern Hopper GPUs that support MIG-style concurrency under constrained, hardware-valid partition shapes. The paper’s core operational claims are therefore about method rather than H200-specific measurement: concurrency improves when jobs are placed on tight partitions, partition selection should preserve future configuration reachability, and dynamic workloads such as LLMs benefit from partition fusion, partition fission, restart-on-OOM, and early restart based on memory prediction (Saraha et al., 25 Aug 2025).

This evidence should be read carefully. It supports the view that H200 participates in the same Hopper software ecosystem of DSM-aware compilation and MIG-aware orchestration, but it does not provide direct H200 quantitative validation for either FlashFuser-style fusion or MIGM-style scheduling. The applicability is architectural, not experimentally established (Huang et al., 15 Dec 2025, Saraha et al., 25 Aug 2025).

6. Measured workload behavior: confidential LLM inference and spectral Ewald summation

The most direct H200 application measurements in the supplied literature come from confidential LLM inference and from GPU-accelerated spectral Ewald summation. In the confidential AI study, the evaluated system used an environment-contained NVIDIA-optimized llama-3.1-8b-instruct model on NVIDIA Triton Inference Server v25.01 with TensorRT-LLM backend v0.17.0, genai-perf, FP8 weights and KV cache, dynamic batching, and a key-value free GPU memory fraction of 0.95; the hardware included a Supermicro SYS-322GA-NR, dual Intel Xeon 6900-series CPUs, and an NVIDIA H200 NVL (141GB HBM3e) connected via PCIe 5.0 (Schambach et al., 30 Jun 2026).

For that LLM inference path, the paper reports that the additional EnclaveX layer adds essentially no inference overhead relative to a native TDX CVM + H200 confidential GPU baseline: “we do not measure any overhead for EnclaveX compared to a native CVM deployment,” with differences described as insignificant and within expected standard deviation. The dominant cost comes instead from running the GPU itself in confidential mode rather than non-confidential mode. Relative to non-confidential native inference, the reported overhead ranges are 35.0% to 62.8% for TPS, 35.2% to 73.9% for TBT, and 11.2% to 38.3% for TTFT, with overhead decreasing as batch size or input size increases because the fixed I/O penalty from encrypted bounce-buffer traffic is better amortized (Schambach et al., 30 Jun 2026).

The same paper measures attestation latency for the H200-enabled TDX setup. Mean latency is 1.1611 s for native TDX + cGPU attestation, 0.0147 s for application-level SCONE attestation, and about 1.1758 s total for EnclaveX, corresponding to an added overhead of around 1.27% (Schambach et al., 30 Jun 2026). In practical terms, the reported conclusion is that full CPU/GPU/application attestation is relatively inexpensive; the substantive runtime tax comes from confidential GPU I/O.

In scientific computing, a performance-portable implementation of the spectral Ewald method for periodic Stokes flow reports direct H200 results across near-field, spreading, FFT, and interpolation kernels. For the near-field P2P interaction, the abstract reports around 73% compute efficiency on NVIDIA H200, compared with 84% on NVIDIA A100, 60% on AMD MI300, 52% on Grace CPU, and 68% on AMD Epyc CPU (Kosmacher et al., 17 Jun 2026). For a full single-GPU run at $N = 4 \times 10^6$, $E = 10^{-9}$, and $s = 224$, the detailed H200 timing breakdown is 239.0 ms for P2P, 47.6 ms for P2G, 48.7 ms for FFT, 20.0 ms for CNV, 10.7 ms for IFFT, and 36.7 ms for G2P, for a total of 402.8 ms. The abstract summarizes overall throughput as approximately 8 million particles per second on a H200 GPU for nine digits of accuracy (Kosmacher et al., 17 Jun 2026).

That paper also identifies a specific algorithmic bottleneck and remedy on H200. A straightforward P2G kernel can become a bottleneck because of atomic contention and irregular access; the proposed hybrid P2G variant achieves up to 16.18× speedup over the baseline GPU implementation (Kosmacher et al., 17 Jun 2026). The authors’ own interpretation is that H200 is especially effective when the implementation matches its machine balance: P2P is compute-efficient, FFT and IFFT benefit from high bandwidth and optimized libraries, and P2G performance depends critically on restructuring the kernel to reduce atomic pressure.

7. H200 as a baseline for the post-Hopper transition

A final strand of literature uses H200 not as the primary object of study but as the baseline against which Blackwell is measured. In that comparison, H200 represents the “traditional Hopper” design point: conventional Hopper memory hierarchy, warp-group-level tensor execution via wgmma, support for FP32, FP16, BF16, FP8, INT8, and FP64, but no native FP6 or FP4 support (Jarmusch et al., 1 Dec 2025).

The Blackwell microbenchmarking paper reports H200 as the reference point in several categories. For tensor-core throughput, H200 values in the reported table are 34.0 TFLOPS for FP64, 378.4 TFLOPS for FP32, 756.9 TFLOPS for TF32, 1513.5 TFLOPS for BF16, 1515.2 TFLOPS for FP16, 3026.9 TFLOPS for FP8, and 3088.4 TFLOPS for INT8; the corresponding B200 numbers are roughly 1.27× higher in the precisions both devices share (Jarmusch et al., 1 Dec 2025). In dense FP64 DGEMM, the paper reports 18.9 TFLOPS on H200 versus 36.30 TFLOPS on B200, while in transformer inference it reports examples such as 28,500 tok/s on H200 versus 45,200 tok/s on B200 for Mistral-7B FP16, and 49,200 tok/s versus 78,400 tok/s for Mistral-7B FP8 (Jarmusch et al., 1 Dec 2025).

The memory comparison is equally explicit. For STREAM Triad, H200 is reported at 2.88 TB/s for a 4GB working set, 2.91 TB/s for 16GB, 4.35 TB/s for 64GB, and 4.38 TB/s for 128GB, while B200 is about 1.71× faster at large working sets. The paper also contrasts H200 global-memory latency of 1000 cycles with B200 TMEM access latency of 420 cycles in cache-miss scenarios, described as a 58% reduction (Jarmusch et al., 1 Dec 2025).

The importance of these comparisons for understanding H200 is not merely competitive ranking. They define the end of a design regime. H200 emerges as a strong Hopper baseline whose algorithmic optimum still depends on warp-group synchronization, shared-memory staging, L2-aware reuse, and conventional mixed-precision modes, whereas Blackwell changes that optimum through TMEM, warp-level tensor instructions, and lower-precision support (Jarmusch et al., 1 Dec 2025). In that sense, H200 occupies a transitional position: it is already a high-bandwidth, large-memory, multi-role accelerator, but one whose programming assumptions are still recognizably Hopper-era.