---
title: 'M0E4 FP4: 4-bit Quantization for Neural Models'
url: https://www.emergentmind.com/topics/m0e4-fp4-quantization-method
type: topic
---

# M0E4 FP4: 4-bit Quantization for Neural Models

The M0E4 FP4 quantization method refers to a class of 4-bit floating-point representations and associated quantization algorithms that target highly efficient training and inference for large language models (LLMs), diffusion transformers, and related neural architectures. This approach relies on an FP4 (4-bit floating-point) data type with zero or minimal mantissa, block-wise scaling, and hardware-tailored dequantization strategies to maximize compute throughput while reducing accuracy loss to minimal levels. Several recent papers detail the construction, implementation, and performance of M0E4 variants, highlighting their utility for both mobile deployment [2403.20041], quantized training [2501.17116], and post-training quantization [2511.04214, 2305.12356].

## 1. FP4 Number Format: M0E4 Variants

The core M0E4 FP4 format adopts a 4-bit floating-point representation, commonly realized as either E2M1 (2 exponent, 1 mantissa, 1 sign) or E4M0/E0M4 (4 exponent, 0 mantissa, 0 or 1 sign). Typical M0E4 layouts include:

- **E2M1 ("standard")**: 1 sign, 2 exponent, 1 mantissa bit; 16 code points representing symmetric values in $\mathcal{S}_{\rm FP4} = \{-6, -4, -3, ..., 0, ..., 6\}$ [2501.17116, 2305.12356].
- **E4M0 ("max exponent")**: 1 sign, 4 exponent, 0 mantissa; used in some mobile GPU implementations [2403.20041].
- **E0M4 ("max mantissa")**: 0 exponent, 4 mantissa; interpreted with groupwise scale and sign [2403.20041].

All M0E4 variants prioritize coarse dynamic range at lowest bit cost, often with special handling for zero and outlier values.

| Format    | Bits             | Range                 | Sign Treatment          |
|-----------|------------------|-----------------------|-------------------------|
| E2M1      | 1S / 2E / 1M     | ±0.5, ±1, ..., ±6     | Per-value               |
| E4M0      | 1S / 4E / 0M     | Wide, stepwise        | Per-value or groupwise  |
| E0M4      | 0E / 4M          | Narrow, dense near 0  | Fixed per group         |

The real value represented by M0E4 FP4 follows:
\[
X_{\rm FP4} = (-1)^s \cdot 2^{e-b} \cdot (1 + \sum_{i=1}^m d_i 2^{-i})
\]
with $b$ an exponent bias, $d_i$ mantissa bits, and $s$ a sign bit [2310.16836].

## 2. Blockwise Quantization and Scaling

Group or blockwise scaling is essential for practical M0E4 deployment. Tensors (weights, activations) are divided into fixed-length blocks (typically 16, 32, or 128 elements). Each block shares a scale parameter (commonly FP8 E4M3) and, in some variants, a shared sign and exponent header [2511.04214, 2403.20041].

- **Block scaling process** [2403.20041]:
  1. Partition tensor into blocks (e.g., $128$ elements).
  2. Determine groupwise statistics: $\max$ and $\min$.
  3. Set scale and bias so all block values fit into one quantization bin.
  4. Quantize per-element mantissa via round-to-nearest, possible rounding-up by $1$ bit.
  5. Store block header (scale, bias, sign) and packed FP4 values.

- **Hardware interpretation**: Dequantization involves merely left-shifting mantissa bits and bitwise OR with header, followed by a reinterpret-cast to FP16; host-device conversions are eliminated [2403.20041].

- **Power-of-two (PoT) block-scaling**: Each block's exponent is chosen as $e_{\rm block} = \lfloor \log_2 (\max |W_{\rm block}|) \rfloor - e_{\max,\rm local}$, ensuring precise packing of statistical data [2511.04214].

## 3. Quantization Algorithms and Outlier Management

Quantization follows a lookup or analytic mapping to closest FP4 code point:
\[
Q(x) = \arg\min_{s \in \mathcal{S}_{\rm FP4}} |x - s|
\]
sign bits, scale factor, and, if needed, bias and zero-point compensate tensor-specific distributions.

- **Differentiable surrogates (for training)**: Quantized training employs a smooth differentiable function $f(x)$ on each quantization bin, with a gently sloped derivative $f'(x)$ instead of STE, to propagate meaningful gradients [2501.17116].
- **Outlier clamp and compensation (OCC)**: Activations are clamped to a quantile threshold $\tau$ (e.g., $99$th percentile); sparse high-magnitude residuals $\Delta Y$ are computed and processed separately in high precision [2501.17116].
- **Per-channel scaling for activations**: To avoid precision loss, activation scaling is often performed per channel or token rather than globally [2310.16836, 2501.17116, 2503.15465].

## 4. Mixed-Precision, Format Selection, and Layerwise Adaptation

No single FP4 layout universally dominates; mixed-precision and per-layer or per-block format selection is a central theme.

- **Mixture-of-Formats Quantization (MoFQ)**: Each layer in a model is quantized using the FP4 or INT4 format that minimally degrades output (as judged by tensor- or model-level MSE) [2305.12356].
- **Hybrid (E1M2, E2M1, E0M4, etc.) layer assignments**: Layerwise grid search or Hessian-weighted optimization arrives at best $(e,m)$ allocations, often with higher exponent (E3M0, E4M0) for early linear layers and more mantissa for deep/final layers [2405.19751, 2503.15465].
- **Blockwise orthonormal rotation (BRQ)**: For PoT scaled formats (MXFP4, M0E4), block-local orthonormal transforms (e.g., Hadamard/Kronecker) are used to redistribute the outlier energy within each quant block, minimizing the impact on blockwise $\max$ and preserving quantization fidelity [2511.04214].

## 5. Implementation and Hardware Efficiency

M0E4 FP4 quantization routinely achieves notable speed and memory gains on modern hardware:

- **Mobile GPUs**: M0E4 dequant involves only two bitwise operations fused into the matmul kernel. Latency for FP16$\times$FP4 compute is up to $1.5\times$ lower on ARM Mali and matches INT4 on Snapdragon Adreno. Quantization accuracy (MAE) is typically up to $4.5\%$ lower than INT4 [2403.20041].
- **NVIDIA GPUs**: NVFP4 and MXFP4 blockwise scaling use FP8 E4M3 for scales. Quantization and dequantization leverage hardware cvt instructions for single-cycle conversion [2512.02010, 2511.04214].

| Hardware      | FP4 Matmul Speedup | Dequantization Overhead |
|---------------|--------------------|-------------------------|
| Mali G720     | $\sim$1.5×         | Zero                    |
| Adreno 750    | 1.0×               | Zero                    |
| NVIDIA Blackwell | $<2\%$ (inference) | Register-only           |

## 6. Empirical Performance and Application Impact

Large language models trained or quantized with M0E4 FP4 recover near-baseline accuracy while drastically reducing memory and compute cost.

- **Training**: FP4 schemes with vectorwise scaling, DGE, and OCC reach training loss and downstream accuracy nearly matching BF16 and FP8 for $>13$B parameter LLMs [2501.17116]. Empirical results:
  - BF16: $54.44\%$ avg accuracy; FP4: $54.95\%$.
- **Post-Training Quantization**: LLMs such as LLaMA-13B achieve $63.1\%$ average zero-shot reasoning (4/4/4 bit-width), only $5.8$ points below full precision and $+12.7$ above prior PTQ SoTA [2310.16836]. MoFQ schemes recover $96$-$98\%$ of FP16 accuracy [2305.12356].
- **Diffusion Transformers**: W4A4 M0E4 quantization leads to sFID degradation $<0.12$ points on ImageNet $256\times 256$, with all INT8 baselines exceeding $25$-$200$ sFID [2405.19751]. Mixup-Sign FP4 with timestep-aware LoRA achieves FID/IS nearly matching FP32 across multiple image synthesis tasks [2505.21591].
- **Adaptive block-level scaling** (Four-Over-Six): Dynamically switching blockwise scale between $4$ and $6$ further recovers $20\%$ of the quantization loss at high values, consistently boosting accuracy to near-BF16 [2512.02010].

## 7. Limitations, Controversies, and Future Directions

Several caveats and research directions arise in the context of M0E4 FP4 quantization:

- **Dynamic range limitation**: With $4$-bit formats, large outliers may be hard-clipped, risking activation collapse unless residual compensation is in place [2501.17116, 2310.16836].
- **Block scaling vs. global rotation incompatibility**: Global orthonormal transforms, effective for INT4, degrade blockwise PoT scaling in MXFP4/M0E4 FP4 contexts [2511.04214]. Local block rotation resolves this but introduces complexity.
- **Format selection overhead**: MoFQ layerwise format selection approximately doubles PTQ calibration time (but no inference overhead) [2305.12356].
- **Precision fallback**: In training, monitoring $\|\nabla L\|/(\sigma_q \sqrt d)$ and switching to higher-precision updates when noise dominates is necessary for convergence [2505.19115].

*This suggests* ongoing research will refine hybrid quantization schedules, residual outlier handling, and hardware abstraction layers for unified FP4 deployment.

---

**Key References:**
- "Optimizing Large Language Model Training Using FP4 Quantization" [2501.17116]
- "LLM-FP4: 4-Bit Floating-Point Quantized Transformers" [2310.16836]
- "Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs" [2403.20041]
- "Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Models" [2305.12356]
- "Block Rotation is All You Need for MXFP4 Quantization" [2511.04214]
- "FP4 All the Way: Fully Quantized Training of LLMs" [2505.19115]
- "Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling" [2512.02010]
- "Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning" [2505.21591]

Source: https://www.emergentmind.com/topics/m0e4-fp4-quantization-method