---
title: 'MXFP4+ (MX+): Advanced 4-Bit Quantization'
url: https://www.emergentmind.com/topics/mxfp4-mx
type: topic
---

# MXFP4+ (MX+): Advanced 4-Bit Quantization

MXFP4+, also referred to as MX+, is a collective term for a suite of enhancements, extensions, and variants of the 4-bit Microscaling Floating-Point (MXFP4) format. These techniques address the critical trade-offs inherent to quantizing activations and weights to ultra low precision (4 bits) in modern neural networks, especially for large language models (LLMs) and vision transformers. MXFP4+ encompasses asymmetric scaling (AMXFP4), precision boosting for outlier elements, oscillation-mitigating quantization strategies, and mixed-precision deployment for improved accuracy and throughput on hardware platforms supporting native MX instructions.

## 1. Fundamental Structure and Variants

The canonical MXFP4 format encodes each element in a tensor block (typically 32 elements per block) using a 4-bit floating-point representation—E2M1, with a shared 8-bit E8M0 block exponent. The MXFP4+ family introduces several key enhancements:

- **AMXFP4 (Asymmetric scaling):** Each 32-element block is equipped with two 8-bit E5M2 shared scales, one for positive and one for negative values. Each element ([s:E:M]=[1:2:1]) is reconstructed according to its sign, exponent, mantissa, and corresponding scale. This layout mitigates high quantization error due to asymmetrically distributed outliers and improves statistical fit for groupwise values [2411.09909].
- **Outlier Repurposing (MX+):** In MX+, the block maximum (outlier) element repurposes its exponent field as additional mantissa bits, effectively expanding its mantissa from 1 to 3 bits while the rest of the block remains at 1+2+1 bits. The index of the outlier is tracked per block (8 bits), and storage overhead is minimal (0.25 bits/element). This innovation significantly reduces the quantization error and loss due to poorly represented extreme values [2510.14557].
- **Oscillation-Reduced Training:** Methods such as Q-EMA (Exponential Moving Average Quantizer) and Q-Ramping (Adaptive Ramping Optimizer) are introduced to stabilize the quantization process during training by damping weight oscillations near quantization thresholds, thus recovering lost convergence and preserving accuracy [2502.20853].
- **Mixed-Precision Assignment:** MicroMix and similar frameworks partition channels or blocks across MXFP4, MXFP6, and MXFP8 formats, guided by quantization error thresholds. The chosen allocation achieves a favorable balance of memory, compute, and accuracy across heterogeneous model components [2508.02343].

The following table summarizes principal MXFP4+ representations and features:

| Variant     | Key Feature(s)                 | Group Scale(s)          | Outlier Handling           | Storage (bits/el.) |
|-------------|-------------------------------|-------------------------|----------------------------|--------------------|
| MXFP4       | Symmetric E2M1, 1× 8-bit E8M0 | 1 per block (absmax)    | None                       | 4                  |
| AMXFP4      | Asymmetric E2M1, 2× 8-bit E5M2| 2 per block (±split)    | None                       | 4.5                |
| MX+         | Outlier ext. mantissa (3 bits)| 1 per block, BM index   | Outlier mantissa boost     | 4.25               |
| MXFP4+ (MicroMix style) | Mixed-format per-channel | 1 per block/channel    | Thresholded assignment     | variable           |

## 2. Quantization and Dequantization Schemes

All MXFP4+ variants operate by normalizing real-valued elements using groupwise shared scales and then rounding to the nearest representable codeword. For standard MXFP4 and MX+ [2510.14557, 2407.01475]:

- **Scale selection:** $s = 2^{\lfloor\log_2\max_{i \in \text{block}} |x_i|\rfloor - b}$ (with bias $b$).
- **Element encoding (non-outlier):** $x_i = \left(1 + \frac{m_i}{2^{b_m}}\right) \times 2^{e_i-\text{bias}} \times s$.
- **Element encoding (outlier, MX+):** $x_{BM} = m_{BM}^{\text{ext}} \times 2^E$ where $m_{BM}^{\text{ext}} = 1 + \frac{M_{BM}}{2} + \frac{X}{8}$ (1 mantissa bit + 2 recycled exponent bits).
- **AMXFP4 encoding:** Each element is rescaled by either $S^+$ or $S^-$ according to its sign, with groupwise exponent rounding and 2-bit mantissa selection for each scale [2411.09909].

Quantization error and rounding are controlled via tie-to-even or stochastic rounding, per the hardware target and training methodology.

## 3. Outlier, Asymmetry, and Oscillation Handling

Quantizing to 4 bits accentuates the vulnerability to extreme activation/weight outliers:

- **Groupwise Outliers:** In the canonical BFP/MXFP4 scheme, the block maximum (BM) dominates the shared scale. Smaller-magnitude elements are compressed into the lowest few representable values, amplifying quantization error.
- **Precision Recovery via Outlier Repurposing (MX+):** The BM element encodes more mantissa bits by repurposing its exponent, rectifying the representational "bottleneck" for outlier elements with negligible impact on the block's overall bitwidth and computation [2510.14557].
- **Asymmetric Scaling (AMXFP4):** Splitting positive and negative sub-blocks allows smaller maximal scales, reduces mean squared error for each sub-distribution, and achieves empirical gains in perplexity and accuracy over symmetric MXFP4 [2411.09909].
- **Oscillation Suppression:** Training in low bitwidth (FP4) regimes can induce persistent weight oscillation—"flip-flop" dynamics—where weights cross static quantization thresholds and fail to converge. Methods such as Q-EMA (anchoring quantized value to a slow-moving average) and Q-Ramping (batch and LR scaling for high-oscillation elements) have been shown to lower oscillation ratio $R_w$, substantially decreasing accuracy loss in vision transformer training settings [2502.20853].

## 4. Hardware Implementations and Integration

MXFP4+ formats have been realized across multiple hardware and software regimes:

- **NVIDIA Blackwell and B200 Architectures:** Native MXFP4 and its MX+ outlier extension are implemented with minimal overhead. BM element identification and extended mantissa datapath add <1% area and sub-1% latency relative to vanilla MXFP4, with a performance uplift due to the accuracy benefits [2510.14557].
- **FPGA Designs:** Open Compute Project MX standards, extended to MXFP4+ (MX⁺), are supported in open-source FPGA IP. Quantization and compute pipelines leverage pipelined Dot and DotGeneral blocks with MX-aware arithmetic, achieving a demonstrable resource/accuracy trade-off on tasks such as ResNet-18/ImageNet [2407.01475].
- **MicroMix Mixed-Precision GEMM:** All MXFP4+ variants are compatible with mixed-precision GEMM kernels, which split tensor channels across MXFP4/6/8 formats as dictated by quantization error thresholds. These kernels issue fused MMA instructions for each group, accumulate in BF16 or FP32, and outperform FP8-based INTGEMMs in both throughput and memory pressure [2508.02343].
- **Software Support:** CUDA, CUTLASS, Triton, PyTorch (Brevitas) host quantization and dequantization kernels designed to handle MXFP4+ packing, extended mantissa logic, BM indexing, and scale computation.

## 5. Empirical Impact and Performance

MXFP4+ techniques consistently demonstrate substantial improvements in both accuracy and efficiency:

- **LLM Inference:** MX+ (outlier repurposing) improves perplexity on LLaMA-3.1-8B from 27.7 (MXFP4) to 9.5 (MX+), and increases zero-shot accuracy for OPT-66B from 35% (MXFP4) to 62% (MX+), matching or exceeding MXFP6 at lower bitwidth [2510.14557].
- **Activation Outlier Scenarios:** AMXFP4 achieves 3–10 point accuracy gains on tasks such as VQA-T, DocVQA, and CSQA, and 3% absolute improvement on MMLU benchmarks compared to symmetric MXFP4 [2411.09909].
- **Vision Transformer Training:** TetraJet with Q-EMA and Q-Ramping halves the MXFP4-induced accuracy drop (improving Top-1 on DeiT-Small from 71.03% to 72.25%; over 50% reduction in the loss relative to full-precision) [2502.20853].
- **Resource Efficiency:** On FPGA, MXFP4+ achieves a Pareto frontier balance: ≈2.6 pp off FP32 for ResNet-18 (QAT), with a 20% LUT area reduction versus FP6 [2407.01475].
- **Mixed-Precision Deployment:** MicroMix (MXFP4+) achieves >95% of FP16 accuracy in zero/few-shot LLM tasks and yields 8–46% kernel-level speedup over TensorRT-FP8, with ~20% memory savings [2508.02343].

## 6. Limitations and Open Research Questions

Several open problems remain in the deployment and further optimization of MXFP4+:

- **Dynamic Group Sizing:** All MXFP4+ variants fix group size at 32; the impact of smaller or adaptive groupings, particularly for skewed modalities (e.g. speech), remains unexplored [2411.09909].
- **Hardware Adoption:** Asymmetric scale and outlier handling require minor, though non-zero, hardware modifications; area and energy impact are small but are not always negligible [2411.09909, 2510.14557].
- **Format Specialization and Rotation:** While MR-GPTQ and blockwise Hadamard rotation can bridge accuracy gaps, their combined effectiveness is format- and group-size-dependent [2509.23202].
- **Unexplored Formats:** Extensions to ternary (FP3) or alternative mantissa/exponent partitioning may yield yet further area/accuracy trade-offs but currently lack empirical evidence [2411.09909].
- **Task and Domain Generality:** Most results are presented for language and vision domains; highly skewed or adversarial data distributions may expose further weaknesses or demand additional enhancements.

## 7. Summary and Outlook

MXFP4+ (MX+) formats collectively represent the state-of-the-art in 4-bit microscaling quantization for neural network deployments, combining custom element layouts (asymmetric, extended mantissa), quantization-aware training, outlier compensation, and hardware-software co-design. By optimally handling the representational limitations of block floating-point in the presence of outliers, MXFP4+ recovers accuracy losses previously considered inherent to FP4, while maximizing performance on emerging hardware such as NVIDIA Blackwell, FPGAs, and custom ASICs. Ongoing research continues to refine block-wise algorithms and hardware micro-architectures, and to broaden empirical validation across new data regimes [2502.20853, 2411.09909, 2510.14557, 2508.02343, 2407.01475, 2509.23202].

Source: https://www.emergentmind.com/topics/mxfp4-mx