FP4 Precision: Low-Bit Efficiency

• FP4 Precision is a 4-bit floating-point format (E2M1 layout) that dramatically reduces memory and bandwidth while preserving exponential dynamic range.
• It incorporates blockwise microscaling and adaptive rounding (round-to-nearest and stochastic rounding) to mitigate quantization noise and maintain accuracy.
• FP4 has proven effective in large-scale training and edge deployments, offering significant improvements in energy, compute, and memory efficiency.

Floating-point four-bit (FP4) precision refers to floating-point representations that fit in exactly four bits per value, typically using an IEEE-style layout with a sign bit, a small number of exponent bits (often 2), and a small number of mantissa bits (often 1). The principal appeal of FP4 is maximally reduced memory and compute bandwidth while retaining the exponential dynamic range that integer formats lack. Recent advances in hardware (e.g., NVIDIA Blackwell tensor cores, custom ASICs, and FPGA designs) now natively support FP4, making aggressive quantized training and inference feasible at scale for LLMs, vision transformers, diffusion models, and edge deployment.

1. FP4 Format Definition and Numeric Properties

FP4 format most commonly takes the form E2M1 (1 sign bit, 2 exponent bits, 1 mantissa), with bias $2^{e-1}-1=1$. Each FP4 datum encodes

$x = (-1)^s \cdot 2^{e-\text{bias}} \cdot (1 + m/2)$

where $s \in \{0,1\}$, $e \in \{0,1,2,3\}$, $m \in \{0,1\}$. The representable magnitudes are $\{0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0\}$ (for both signs). Subnormals use $e=0$, and reserved exponent bits ($e=3$) encode Inf/NaN. Larger variants, such as E4M3, E8M0, and UE5M3, increase exponent or mantissa width for specialized scaling blocks or research into optimal trade-offs (Hu et al., 22 Sep 2025).

FP4 offers drastically reduced representational granularity (maximum relative quantization error ≈ 25%), but blockwise scaling (e.g., FP8/E4M3 or FP16/E8M0 scales per 16–32 elements) amplifies its usable dynamic range. Non-uniform spacing—50% at the lowest magnitude, up to wide steps near the maxima—enables effective mapping of bell-shaped or long-tailed distributions, which is advantageous over uniform INT4 quantization in modern deep networks (Liu et al., 2023, Liu et al., 30 May 2024).

2. Blockwise Quantization and Microscaling

FP4 is almost always used with blockwise scaling—each small block of values (size 16, 32, occasionally 128) shares a high-precision scale factor. The quantization pipeline for block $b$ (e.g., NVFP4 or MXFP4) is

$$q_{b, i} = \text{Round}_\text{FP4}\left(\frac{x_{b, i}}{s_b}\right) \qquad \hat{x}_{b, i} = s_b \cdot q_{b, i}$$

Here $s_b$ is stored in a high-precision format (FP8, FP16), typically optimized to align $|x_{b,i}|$ with the largest FP4 code point (e.g., $s_b = \max_j |x_{b, j}| / 6$). Blockwise scaling ("microscaling") fits local dynamic range and substantially mitigates the quantization noise arising from FP4's coarse spacing (NVIDIA et al., 29 Sep 2025, Chmiel et al., 25 May 2025, Liu et al., 4 Aug 2025).

Adaptive block scaling strategies, such as Four Over Six (4/6), further minimize error by evaluating multiple candidate scales (e.g., scaling to $6$ vs. scaling to $4$) and selecting the scale yielding the lowest MSE (Cook et al., 1 Dec 2025). This is particularly useful for values near each block's maximum, which otherwise suffer large quantization jumps due to missing codepoints.

3. Rounding Modes and Error Mitigation

The stability and accuracy of FP4 quantization depend critically on rounding modes and quantization error control:

• Forward Pass: Round-to-nearest is preferred for deterministic computation and minimal bias.
• Backward Pass/Weight Updates: Stochastic rounding (SR) is essential, providing unbiased expectation and preventing small gradients from vanishing; each value is probabilistically assigned to the nearest FP4 codepoints based on its fractional distance (Chmiel et al., 25 May 2025, NVIDIA et al., 29 Sep 2025).

Stochastic rounding suppresses bias and enables theoretically stable convergence, but quantization noise dominates if the gradient norm drops below $\sqrt{3}$ times the per-coordinate noise standard deviation. Above this threshold, FP4 training is empirically loss-competitive with BF16 and FP8 (Chmiel et al., 25 May 2025).

Blockwise Hadamard transforms (random sign-flip orthogonalizations), outlier detection, and two-dimensional scaling (matching quantization blocks between forward and backward passes) are further adopted to mitigate outlier inflation and preserve unbiased gradient propagation (NVIDIA et al., 29 Sep 2025, Chen et al., 31 Oct 2025, Liu et al., 30 May 2024).

4. FP4 in Large-Scale Training and Mixed-Precision Schemes

FP4 enables "fully quantized training" (FQT), where weights, activations, and gradients are all stored and processed in FP4, often with selective retention of high-precision in critical layers to avoid loss divergence (Chmiel et al., 25 May 2025, NVIDIA et al., 29 Sep 2025). Mixed-precision systems also exploit FP4 for the majority of computation while assigning FP8/BF16 only to blocks identified as high-sensitivity by Fisher-weighted metrics or outlier statistics (Hooper et al., 19 Apr 2025, NVIDIA et al., 29 Sep 2025, Chen et al., 31 Oct 2025, Chen et al., 28 Feb 2025).

Illustrative training workflow:

1. Forward GEMM (matrix multiply): Inputs quantized to FP4 via microscaling (e.g., NVFP4, MXFP4). Round-to-nearest.
2. Backward GEMM: Gradient inputs quantized with stochastic rounding.
3. Update GEMM: Stochastic rounding for both activations and gradients.
4. Selective layers: Some layers retained in BF16 for stability (typically initial and final blocks).
5. Optimizer states: Kept in FP8/FP16/BF16; moments and learning rates computed in higher precision.
6. Outlier Handling: Hadamard transforms (orthogonalization), sparse compensation for clamped activations, and per-channel formatting (NVIDIA et al., 29 Sep 2025, Chen et al., 31 Oct 2025, Wang et al., 28 Jan 2025).

Recent algorithms (Quartet, TetraJet-v2) combine unbiased double-block quantization, oscillation suppression (OsciReset, EMA Quantizer, Q-Ramping), and explicit outlier control to close the accuracy gap to full precision by more than 50% (Castro et al., 20 May 2025, Chen et al., 31 Oct 2025, Chen et al., 28 Feb 2025). The low-precision scaling law derived in Quartet predicts optimal accuracy-vs-compute trade-offs for FQT, informing layer and bitwidth choices.

5. Hardware Implementations and Performance

Modern hardware (NVIDIA Blackwell, Intel Gaudi2, custom ASICs, FPGAs) support native FP4 via dedicated tensor-core instructions (e.g., tcgen05.mma), blockwise scale operands, and SIMD MAC datapaths optimized for tiny formats (Jarmusch et al., 14 Jul 2025, Chaudhari et al., 18 Aug 2025, Lokhande et al., 12 Oct 2025). A typical hardware pipeline features:

• Packing: Each FP4 value occupies 4 bits; two values per byte.
• Blockwise scale storage: FP8/FP16/E8M0 per 16–32 values; scale factors enable dynamic range extension.
• Multiply-accumulate: Internally performed in full FP32, preserving partial sum precision.
• Latency and throughput: Blackwell achieves up to 11 TFLOPS/SM at ≈1.2 cycles true latency and 4× FP16 throughput; FPGA accelerators yield up to 684.5 GOPS at 8.12 W on translation (Jarmusch et al., 14 Jul 2025, Lokhande et al., 12 Oct 2025, Chaudhari et al., 18 Aug 2025).
• Energy/area: FP4 MAC units consume ≈14 pJ/op and occupy 0.016 mm² per MAC, representing substantial area and energy savings over FP8/BF16 (Chaudhari et al., 18 Aug 2025).

FP4 hardware co-designs further support mixed-precision VMAC engines, dot-product units with format selection, rapid activation quantization, and per-block metadata flags to steer precision assignment (Hooper et al., 19 Apr 2025, Chaudhari et al., 18 Aug 2025). Practical guidelines stress memory layout (channel-ordered, block-packing), shared-memory pre-staging, register occupancy management, and explicit kernel designation for FP4 operations (Jarmusch et al., 14 Jul 2025).

6. Empirical Results and Use Cases

FP4 is empirically validated for:

• LLM Pretraining and Inference: Comparable downstream accuracy to FP8/BF16 across MMLU, ARC, PIQA, HellaSwag, LAMBADA, with typical loss gaps <1% (NVIDIA et al., 29 Sep 2025, Chen et al., 31 Oct 2025, Liu et al., 2023).
• Diffusion and Vision Transformers: W4A4 hybrid FP4 quantization yields ≤0.12 sFID increase on ImageNet; top-1 accuracy drops reduced by more than 50% (e.g. Q-EMA and Q-Ramping in Oscillation-Reduced MXFP4 Training) (Liu et al., 30 May 2024, Chen et al., 28 Feb 2025).
• XR and Edge AI: Efficient MAC engines in XR-NPE deliver 42% reduced area, 38% reduced power, and only ~0.7pp accuracy drop in VIO tasks vs. FP32 (Chaudhari et al., 18 Aug 2025, Lokhande et al., 12 Oct 2025).
• Post-Training Quantization (PTQ): Mixed FP4/FP8 memory layouts enable up to 39% weight memory savings and 14% energy savings with <1% perplexity degradation, outperforming pure INT4 PTQ (Hooper et al., 19 Apr 2025, Wu et al., 2023).
• Low-resource translation: Bhasha-Rupantarika achieves 4.8× speedup and only 1.2 BLEU loss, making real-time translation on FPGAs practical (Lokhande et al., 12 Oct 2025).

The primary limitations are residual degradation for extremely outlier-heavy layers, convergence stalls when using all-FP4 below the noise threshold, and a need for strategic mixed-precision or outlier management in large models (especially >7B parameters).

7. Practical Guidelines and Research Directions

• Blockwise microscaling with fine-grained precision assignment is essential for error control—avoid global scaling or naive tensor-wide FP4 quantization (Chmiel et al., 25 May 2025, Liu et al., 4 Aug 2025).
• Apply stochastic rounding on all backward passes; round-to-nearest forward for unbiased training and reduced bias.
• Oscillation suppression (OsciReset, EMA Quantizer, Q-Ramping) and explicit outlier control (per-channel formatting, Hadamard transformation) reduce noise and stabilize long runs (Chen et al., 28 Feb 2025, Chen et al., 31 Oct 2025).
• Monitor gradient norm vs. quantization noise to trigger fallback to higher precision or brief quantization-aware fine-tuning when necessary (Chmiel et al., 25 May 2025).
• Use double-block quantization (global+block scaling) for large tensors to avoid scale overflows and maintain quantization consistency between FPROP and BPROP (Chen et al., 31 Oct 2025, NVIDIA et al., 29 Sep 2025).
• FP4 is most advantageous when memory, bandwidth, and energy constraints dominate; its hardware and algorithmic ecosystem now support fully quantized training and inference across transformer, diffusion, and vision domains.

For future research, extensions toward FP6/FP8 hybrids, further non-uniform scaling formats (UE5M3), hardware support for conditional update scheduling, unified oscillation-aware quantizers, and more sophisticated outlier detection are actively explored (Hu et al., 22 Sep 2025, Chen et al., 28 Feb 2025, Chen et al., 31 Oct 2025). Empirically, the Quartet scaling law and TetraJet-v2 benchmarks suggest that the accuracy–efficiency Pareto frontier supports native FP4 as a strong contender for next-generation training and deployment at scale.