---
title: NVIDIA FP4 Quantization Algorithm
url: https://www.emergentmind.com/topics/nvidia-fp4-nvfp4-quantization-algorithm
type: topic
---

# NVIDIA FP4 Quantization Algorithm

NVIDIA FP4 (NVFP4) Quantization Algorithm

NVIDIA FP4 (NVFP4) quantization is a hardware-native, block-microscaled, 4-bit floating-point scheme implemented in NVIDIA Blackwell-class GPUs to accelerate large language model (LLM) training and inference. NVFP4 enables aggressive precision reduction for weights, activations, and gradients, yielding substantial increases in throughput and substantial reductions in memory and energy consumption, while maintaining near-lossless accuracy on large-scale transformers. NVFP4 achieves this by using carefully engineered per-block scaling, unbiased stochastic rounding, algorithmic outlier suppression, and adaptive scaling extensions, as described in recent works including TetraJet-v2 [2510.27527], Four Over Six (4/6) adaptive scaling [2512.02010], and a suite of benchmark studies and production frameworks.

## 1. Format Definition: E2M1 Floating-Point and Block Microscaling

NVFP4 encodes each value as an E2M1 4-bit floating-point number, consisting of:

- 1 sign bit
- 2 exponent bits (bias=1)
- 1 mantissa bit

The representable value set is
\[
\mathrm{FP4Values} = \{0, \pm 0.5, \pm 1, \pm 1.5, \pm 2, \pm 3, \pm 4, \pm 6\}
\]
yielding a dynamic range of $\pm6$ [2510.27527, 2512.02010].

To address the limited representational capacity of FP4, NVFP4 attaches an 8-bit E4M3 floating-point scale to each consecutive block of 16 elements (block size $B=16$), known as block microscaling. The E4M3 scale is capable of encoding scaling factors in $[-448, 448]$, so each block may cover a dynamic range of approximately $\pm 2\,688$ after scaling. For full-tensor scaling, an additional FP32 scale per tensor may be applied to maximize representability in high-dynamic-range data [2509.25149, 2505.19115, 2510.25602].

**Comparison to MXFP4:** MXFP4 uses blocks of 32, and an 8-bit E8M0 (integer-only) scale, resulting in greater quantization error due to less precise scaling [2510.27527, 2509.23202].

## 2. Block-wise and Double-block Scaling Quantization Schemes

The NVFP4 quantization process proceeds as follows for each block of 16 values $X = [x_1, \ldots, x_{16}]$:

1. **Block scale selection:** 
   \[
   s = \frac{\max_i |x_i|}{6}
   \]
   Quantize $s$ to E4M3, denoted $s_q$.

2. **Quantize each element:** 
   \[
   x_i^{FP4} = \operatorname{round}_{FP4}\left(\frac{x_i}{s_q}\right)
   \]
   where the rounding is to nearest-representable E2M1 value.

3. **Storage:** Store the 16 4-bit FP4 values plus the single E4M3 scale.

**Double-block scaling** (TetraJet-v2): For improved utilization of the scaling range,
- An outer scale $S_{\rm global}$ is computed over a larger block (e.g., 128 elements) to compress the global dynamic range.
- Each inner block of 16 then has an additional local scale $S_{\rm block}$.
- Final quantization applies both scales:
  \[
  P_i = \operatorname{round}_{FP4}\left(\frac{x_i/S_{\rm global}}{S_{\rm block}(\lfloor i/16 \rfloor)}\right)
  \]
This hierarchical scaling provides better coverage of both global and local outlier variation [2510.27527].

**Unbiased SGD:** Stochastic rounding ($\operatorname{round}_{SR}$) is used in the backward pass, while round-to-nearest is used in the forward pass. This guarantees $\mathbb{E}[\widehat X] = X$, enabling unbiased stochastic gradient descent (SGD) essential for training stability [2510.27527, 2505.19115, 2509.25149, 2501.17116].

## 3. Outlier Handling: Random Hadamard Transform and High-Precision Retention

Quantization error in NVFP4 is dominated by outliers and groupwise maxima, especially in transformer activations and gradients. Distinct approaches have been proposed:

- **Random Hadamard Transform (RHT):** Before quantization, multiply the relevant tensor (typically gradients or activations during backpropagation) by a random (diagonal-signed) Hadamard matrix. This spreads outlier mass across the block, reducing per-channel variance and making the distribution better suited to the limited E2M1 grid [2510.27527, 2509.25149]. After quantization and GEMM, the inverse transform is applied. RHT is highly effective for block sizes $d = 16$, and overhead is minimal in hardware.
- **Static high-precision retention:** Persistent outlier channels, detected via offline profiling, are kept in higher precision (e.g., BF16/FP8), while the remainder use NVFP4. This is applied to a small fraction (e.g., the top $p\%$ channels in a tensor) [2510.27527, 2509.25149, 2504.14152].

Table: Summary of Outlier Mitigation Components

| Method          | Source            | Apply to     | Principle                                  |
|-----------------|-------------------|--------------|--------------------------------------------|
| RHT             | [2510.27527]      | Gradients    | Spread outlier variance, bound quant noise |
| Static Retention| [2510.27527]      | Forward/Back | Profiled high-norm channels kept in HF     |

## 4. Oscillation Suppression and Rounding Strategies

NVFP4 quantization at extremely low learning rates can induce bin-boundary oscillations ("ping-ponging") of weights between adjacent FP4 bins, degrading convergence.

- **OsciReset** (TetraJet-v2): This method tracks, per weight, the ratio of FP4 quantization jumps to the true weight movement over a sliding window:
  \[
  \mathrm{OsciRisk}(w) = \frac{\sum |Q(w^{(t)}) - Q(w^{(t-1)})|}{\sum |w^{(t)} - w^{(t-1)}|}
  \]
  If this risk exceeds a threshold, the weight is recentered at the midpoint of its current FP4 bin [2510.27527].

**Rounding strategy and SGD unbiasedness:** NVFP4 implementations employ round-to-nearest (RTN) in the forward path (minimize representational error) and stochastic rounding (SR) in the backward path (unbiased gradient estimates). Empirically, SR on gradients alone suffices for stable training; activations and weights use RTN [2505.19115, 2510.27527, 2509.25149].

## 5. Adaptive Scaling: Four Over Six (4/6) and MSE-Optimal Schemes

Block scaling induces particularly severe quantization error for values just below the block maximum due to the coarse E2M1 step size between representable values (e.g., between FP4 4 and 6, the step size is 2). This leads to relative error spikes at high-magnitude inputs.

- **Four Over Six (4/6) Adaptive Scaling:** For each block, both $s_6 = \max |X|/6$ and $s_4 = \max |X|/4$ are considered. The block is quantized twice (once to lattice $\{\pm 0, ..., \pm 6\}$, once to $\{\pm 0, ..., \pm 4\}$). The scale yielding lower mean-squared error (MSE) is selected for actual storage:
  \[
  E^{(N)} = \frac{1}{n}\sum_i \left[ s_N \cdot Q_{FP4}(X_i/s_N) - X_i \right]^2, \quad N \in \{4,6\}
  \]
  [2512.02010].

This modification efficiently improves representation of near-maximum values, prevents divergence in both pretraining and post-training quantization scenarios for large-scale architectures, and can be implemented as a minor extension of the standard kernel. Empirical results show universal improvements for common LLMs and substantial reductions in perplexity and accuracy gaps [2512.02010].

## 6. Empirical Performance and Model-Scale Impact

NVFP4 quantization, when integrated with advanced scaling, unbiased rounding, outlier-mitigation, and oscillation suppression, yields practical large gains for LLM training and inference.

- **TetraJet-v2**: On OLMo-2 LLMs up to 370M parameters and 200B tokens, NVFP4 with double-block scaling, RHT, OsciReset, and OutControl yields validation perplexities close to BF16, and downstream accuracy drops under 1% [2510.27527].
- **Four Over Six**: Applying 4/6 adaptive scaling universally prevents training divergence and closes the loss gap to BF16 on transformer and hybrid models, reduces WikiText-2 perplexity by 3–7%, and boosts downstream accuracy by up to 1 point with negligible runtime overhead [2512.02010].
- **End-to-end LLM training**: Training with NVFP4 (and stochastic rounding + RHT + partial high-precision layers as needed) achieves within 1–2% of FP8/BF16 baseline accuracy, with stable convergence on models of scale up to 12 B and 10 T tokens [2509.25149, 2505.19115]. MXFP4 requires up to 36% more tokens to reach comparable loss [2509.25149].
- **Inference speedups**: Direct NVFP4 execution on Blackwell GPUs delivers up to $3.6\times$ speedup (single linear layer), and $2.0$–$4.0\times$ end-to-end speedup on LLM inference [2509.23202, 2508.02343].

## 7. Hardware, Mixed-Precision, and Comparative Context

NVFP4 is directly supported by Blackwell-class NVIDIA GPUs and is the canonical low-bit format for 4-bit microscaling in the hardware and software ecosystem. Key low-level details include:

- **Hardware implementation:** Each NVFP4 block of sixteen 4-bit E2M1 values is stored with an E4M3 scale. Processing units perform blockwise dequantization and accumulation in FP16/FP32, with tightly integrated rounding and scale application. Optimized CUDA and CUTLASS kernels fuse quantization, dequantization, and matrix multiplication while utilizing FP4 instruction sets [2508.02343, 2509.23202].
- **Comparison to INT4 and MXFP4:** For small block sizes, NVFP4 typically outperforms INT4 and MXFP4 on both accuracy and stability, barring advanced outlier mitigation for INT4 (e.g., Hadamard rotations) [2510.25602, 2509.23202]. For hardware area and energy, NVFP4 improves over larger FP scalar formats but is moderately less efficient than NVINT4.
- **Mixed-precision integration:** For robustness, NVFP4 is frequently deployed with selective retention of FP8/BF16 in a small set of high-impact layers or persistent outlier channels [2510.27527, 2509.25149, 2504.14152]. These strategies allow >30% weight memory reduction at <1% accuracy loss on major LLMs.
- **Software ecosystem and kernels:** Production kernels support fused rotation, scale search (MSE-optimized grids), static activation permutation, and blockwise quantization on NVFP4. Recent quantization frameworks (e.g., MR-GPTQ, TetraJet-v2, MicroMix) specifically optimize for FP4 hardware constraints and accuracy restoration [2509.23202, 2510.27527, 2508.02343].

---

In summary, the NVIDIA FP4 (NVFP4) quantization algorithm—by combining small-block E2M1 floating-point values, high-resolution block E4M3 scaling, unbiased stochastic quantization, algorithmic oscillation suppression (OsciReset), adaptive scaling (4/6), and hybrid outlier control—restores near-full-precision convergence and accuracy for LLM training and inference at 4-bit cost. The format is now foundational in Blackwell platforms and informs the algorithm–hardware co-design of next-generation low-precision AI accelerators [2510.27527, 2512.02010, 2509.25149, 2505.19115, 2509.23202].

Source: https://www.emergentmind.com/topics/nvidia-fp4-nvfp4-quantization-algorithm