---
title: MXFP8-E4M3 Floating-Point Format
url: https://www.emergentmind.com/topics/mxfp8-e4m3-floating-point-format
type: topic
---

# MXFP8-E4M3 Floating-Point Format

MXFP8-E4M3 Floating-Point Format

MXFP8-E4M3 is a block-scaled, non-IEEE, 8-bit floating-point format consisting of 1 sign bit, 4 exponent bits, and 3 mantissa (fraction) bits, with an exponent bias of 7. It forms the canonical “microscaling FP8” (“MXFP8-E4M3”) data type, now widely adopted for efficient quantization and training of large-scale neural networks, notably large language models (LLMs) and vision transformers, particularly on hardware supporting fast, low-width floating-point operations such as NVIDIA H100/Blackwell architectures and RISC-V extensions. The format's design balances dynamic range and precision, provides native hardware support for blockwise dot products, and supports near-lossless quantized inference and training at reduced memory and compute cost.

## 1. Bit Layout and Numeric Encoding

MXFP8-E4M3 comprises 8 bits partitioned as:

- 1 sign bit (s): Determines the sign.
- 4 exponent bits (E): Encoded exponent with bias 7.
- 3 mantissa bits (m): Encodes the significand fraction.

The encoded value is
- For $1 \leq E \leq 14$ (normalized):  
  \[
  x = (-1)^s \cdot 2^{E-\text{bias}} \cdot \left(1 + \frac{m}{2^3}\right)
  \]
- For $E=0,$ $m>0$ (subnormal):  
  \[
  x = (-1)^s \cdot 2^{1-\text{bias}} \cdot \frac{m}{2^3} = (-1)^s \cdot m \cdot 2^{-9}
  \]
- For $E=0,m=0$: Encodes $\pm 0$.
- $E=15$ with $m=0$: $\pm\infty$ (in some implementations, only NaN).
- $E=15$, $m>0$: NaN.

The representable range for normalized numbers is $2^{-6} \leq |x| < 240$ with a machine epsilon $\varepsilon=0.125$ at $|x|=1$; subnormals extend the minimum to $2^{-9}$ [2307.09782, 2104.07329, 2601.09555].

| Field         | Bit-width | Interpretation               | Range/Values      |
|:--------------|:---------|:-----------------------------|:------------------|
| Sign $(s)$    | 1        | $0$ for $+$, $1$ for $-$     | $\{0,1\}$         |
| Exponent $(E)$| 4        | Encoded, bias $=7$           | $[0,15]$          |
| Mantissa $(m)$| 3        | $0$ for integer, $>0$ fraction| $[0,7]$           |

## 2. Blockwise Scaling (Microscaling) and Conversion

MXFP8-E4M3 is always paired with a block-shared scale (typically $K=32$ contiguous values share one scale), stored as an 8-bit power-of-two unsigned exponent (e.g., UE8M0 as per Blackwell GPUs). The encoding-decoding proceeds as follows [2506.08027, 2411.03149, 2505.13159]:

- For input tensor $V$ partitioned into blocks $b$ of length $K$:
  1. Compute $a_{\max} = \max_i |V_b[i]|$.
  2. Compute scale exponent ${\sf expX} = \lceil \log_2(a_{\max}/\text{destmax}) \rceil$, clamped to $[-127,127]$.
  3. Write scale as $s_b = {\sf expX} + 127$ (in UE8M0).
  4. Per element: $Q_i = \operatorname{RN}(V_b[i]/X)$; clamp to E4M3 limits.
  5. Store quantized block $(Q_0,\ldots, Q_{K-1}, s_b)$.

On dequantization or hardware conversion, $Q_i$ is expanded to real value using the per-block scale:  
\[
V_b[i] = Q_i \cdot X = (-1)^{s} 2^{E-\text{bias}} \left(1 + \frac{m}{8}\right) \cdot 2^{\text{expX}}
\]

Pseudo-code and specific conversion details in [2506.08027, 2411.03149, 2601.09555].

## 3. Dynamic Range, Precision, and Comparison

The normalized representable range is $[2^{-6}, 240]$; with blockwise scaling, aggregate dynamic range is greatly expanded ($2^{-9}$ to $240 \cdot 2^{127}$ with power-of-two block scale). The mantissa provides 4 bits of precision (unit in the last place, ULP, $2^{-3}$). Subnormal values fill the underflow gap, enabling contiguous quantization even for small values. 

Compared to E5M2 (5 exponent, 2 mantissa) or INT8:
- E4M3 maintains higher mantissa precision but a smaller exponent range.
- For distributions dominated by outliers, E4M3 (thanks to the exponent bits) outperforms integer and E5M2 variants in quantizing LLM activations and weights with heavy-tailed or spiky statistics [2307.09782, 2208.09225, 2601.09555].
- Empirical results on DNNs (ResNet, VGG) show that E4M3 achieves $<0.5\%$ accuracy drop vs. FP32 with proper bias selection and block scaling [2104.07329].
- On LLMs, MXFP8-E4M3 quantization yields $\leq 0.3\%$ average accuracy drop and $\leq 5\%$ relative perplexity increase in the worst case under post-training quantization (W8A8) [2601.09555].

## 4. Deployment in Neural Network Quantization

For quantized LLMs and vision models, MXFP8-E4M3 is applied as follows:

- **Activations:** Per-token or per-tensor affine quantization using the E4M3 grid, with scales calibrated to fit maximum magnitude into $[-240,240]$ [2307.09782].
- **Weights:** Fine-grained group quantization (FGQ), typically with block sizes of 32 and dyadic (power-of-two) scales to facilitate efficient hardware mapping. Converted to E4M3 at runtime for hardware compatibility.
- **Post-Training Quantization (PTQ):** Blockwise (size 32) scaling, round-to-nearest-even, various algorithms (GPTQ, MR-GPTQ, FlatQuant, SmoothQuant) adapt readily. Rotational transform methods (QuaRot, SpinQuant) do not improve, and sometimes harm, performance in low-bit MXFP [2601.09555].
- **Scaling Heuristics:** Ceil(log2)-based rounding of the scale ensures all values are in-range, minimizes overflows, and stabilizes training. In INT4/FP4+FP8 hybrid settings, scale alignment (nearest power-of-two, group-wise dyadic) enables fast reinterpret casts and high TensorCore throughput [2506.08027, 2307.09782].

Empirically, E4M3 quantization enables quantized pre-training and inference for LLMs at double the speed of BF16, while maintaining matching perplexity and accuracy with models up to 8B or even 16B parameters [2506.08027, 2601.09555].

## 5. Hardware Implementations and Architectures

MXFP8-E4M3 enjoys broad hardware support, both in commercial GPUs and open-source cores:

- **NVIDIA H100/Blackwell TensorCores:** Native E4M3 arithmetic kernels perform MXFP8 matmuls and fused-multiply operations at full speed; block-wise scaling is managed by software with per-block scales, and dyadic-intensity optimizations facilitate ultra-low-latency FP4→FP8 conversion [2307.09782, 2506.08027].
- **RISC-V MXDOTP Extension:** Dedicated instructions implement fast blockwise dot-products with packed 8-bit E4M3 operands and power-of-two block scales, achieving $>350$ GFLOP/W at 1 GHz, 25× higher energy efficiency than software-emulated FP8 [2505.13159].
- **FPGA Block Converters:** Conversion units for FP32→MXFP8-E4M3 efficiently process batches of 32 values, hierarchically extracting the scale and mapping to E4M3 [2411.03149]. The design partitions LUTs between max-exponent trees and quantization logic, supporting high-throughput, low-area implementations.
- **FPnew Transprecision FPU:** Parameterized multiprecision units, supporting E4M3 as a native format, achieve up to 2.95 TFLOP/s/W in 8×SIMD mode and 14.8 GFLOPs single-core at 923 MHz [2007.01530]. All IEEE-754 rounding and exception handling is supported.

| Hardware            | Block Size | Throughput (GFLOPs) | Energy Efficiency (TFLOPs/W) |
|---------------------|------------|---------------------|------------------------------|
| NVIDIA H100/Blackwell| 32        | 2× BF16 GEMM        | Not specified                |
| RISC-V MXDOTP (8-core) | 8      | 102                 | 0.356                        |
| FPnew (Ariane core) | 8          | 14.8                | 1.25 (up to 2.95)            |
| FPGA (xcvu440)      | 32         | N/A                 | N/A                          |

*Throughput measured at baseline voltage/frequency; see respective papers for details [2505.13159, 2007.01530, 2411.03149, 2506.08027, 2307.09782]*

## 6. Algorithmic and Practical Considerations

- **Rounding:** “Round-to-nearest-even” is standard; saturating to the E4M3 representable minimum or maximum as necessary [2601.09555, 2506.08027].
- **Best Practices:**
  - Block size $K=32$ for scaling.
  - Use “ceil(log2)” in scale computation to bound all values in range.
  - Calibration set size: $512$–$1024$ samples per layer suffices.
  - Prefer error-compensation or affine PTQ algorithms; avoid rotational transforms for E4M3 [2601.09555].
- **Low-Rank Compensation (LoRC):** Adding a blockwise low-rank correction to quantized weights effectively restores subnormal precision; a single $r=4$–$8$ correction can close most of the quality gap for small models [2307.09782].
- **Format Selection:** For models/activations with Gaussian distributions and negligible outliers, E5M2 may give marginally higher SQNR, but E4M3 is superior when LLMs or ViTs present heavy tails, outliers, or large activation spikes [2208.09225].
- **Per-Layer or Per-Group Flexibility:** Allowing bias or field widths to be tuned per-layer can recover several tenths of a percent in accuracy without retraining [2104.07329].

## 7. Empirical Results and Application Scope

Benchmarking across LLM, vision, and mixture-of-expert models consistently finds:

- W8A8 MXFP8-E4M3 delivers near-lossless quantization in LLMs, with $<1\%$ degradation across post-training quantization methods. W4A8 is feasible with refined PTQ but exhibits 1–3% additional loss; W4A4 is not recommended [2601.09555].
- On DNN classifiers (ResNet-50, VGG-16), E4M3 layer-wise flexible quantization reduces top-1 error to within $0.3\%$ of FP32 [2104.07329].
- On H100/Blackwell hardware, MXFP8-E4M3 enables all-core 8-bit inference and training with no significant accuracy loss (PPL curve overlap with BF16), even for multi-billion-parameter LLMs over extended training horizons [2506.08027].
- Hardware-efficient deployment: Zero-overhead in dot product and matrix-multiplication kernels (fast as INT8 or faster, but more accurate in the presence of outliers).

MXFP8-E4M3, by balancing exponent and mantissa bits and leveraging blockwise scaling, provides a broadly applicable, hardware-native, and quantization-efficient format for modern neural network inference and training pipelines [2506.08027, 2505.13159, 2307.09782, 2601.09555].

Source: https://www.emergentmind.com/topics/mxfp8-e4m3-floating-point-format