---
title: Microscaling (MXINT) Formats Overview
url: https://www.emergentmind.com/topics/microscaling-mxint-formats
type: topic
---

# Microscaling (MXINT) Formats Overview

Microscaling (MXINT) Formats

Microscaling (MXINT) formats are a family of block-scaled, low-bitwidth quantization schemes for model weights, activations, and gradients, designed to enable highly efficient large-scale deep learning on modern hardware accelerators. They combine a per-block, power-of-two scaling factor ("microscale") with narrow-width element representations (e.g., INT8, various FP8/FP6 variants) to simultaneously achieve wide dynamic range, high compute/memory density, and minimal accuracy loss during both inference and training. These formats are standardized by the OCP MX working group and are natively supported in recent GPU (NVIDIA Blackwell), FPGA, and NPU architectures [2506.08027].


## 1. Core Structure and Definitions

In MXINT, tensors are partitioned into blocks (typically size $k=32$). Each block is represented by a single shared scale factor $s$—always a power of two, encoded via the 8-bit E8M0 format—and $k$ elements each in a narrow integer or low-bit float format (e.g., INT8, E4M3, E5M2, etc). The canonical INT version, MXINT8, stores each element as an 8-bit signed integer:

For a block $V = (V_1, \ldots, V_k)$:
- $m = \max_i |V_i|$,
- $e_\text{shared} = \lfloor \log_2(m) \rfloor - 7$,
- $s = 2^{e_\text{shared}}$,
- $q_i = \operatorname{clamp}(\operatorname{round}(V_i / s), -127, 127)$,
- $\hat V_i = s \cdot q_i$.

The block representation is
- $k$ integer elements ($q_i$): usually INT8, but also explored for INT4, INT6, E2M3, etc,
- 8-bit exponent $s$ (per block).

The same shared scaling pattern can be applied to low-bit floating-point element layouts, most commonly E4M3 (sign:1, exponent:4, mantissa:3), E5M2, and down to E2M1 (4 bits/element).


## 2. Quantization and Dequantization Algorithms

Microscaling quantization consists of three key steps applied per block:
1. **Block scaling:** Find $m = \max_i |V_i|$, set $s = 2^{\lceil \log_2 (m / \mathrm{dest\_max}) \rceil}$ for floats, or $s = m / Q_\text{max}$ (power-of-two rounded up) for integers, with $Q_\text{max} = 127$ for INT8.
2. **Element-wise quantization:** $q_i = \operatorname{clamp}(\operatorname{round}(V_i / s), Q_\text{min}, Q_\text{max})$.
3. **Dequantization:** $\hat V_i = s \cdot q_i$.

Crucially, the scale $s$ is always a power-of-two, which enables efficient hardware implementation via shift/scaling rather than general multiplication [2310.10537][2506.08027].

In floating-point MX formats (e.g., E4M3 or E5M2), quantization proceeds as
- $V_i \to \operatorname{QuantizeToMXFP}(V_i / s)$
where elementwise quantization follows the IEEE-754 layout with reduced exponent/mantissa.

Block scales are recomputed for every block access (no "scale update frequency" hyperparameter). Rounding up of exponent (ceil) is critical to avoid out-of-range errors and guarantee no overflow during quantization [2506.08027].

For training, various enhancements are used: symmetric clipping for INT8 (mapping $q \in [-127,127]$ to eliminate negative bias in gradient flows), optional Hadamard rotation for INT4 robustness, and first- and second-order quantization-aware training (QAT) methods [2510.25602][2405.07135].


## 3. Precision Selection, Hybrid and Metadata-Enhanced Schemes

The choice of element type (INT8, E4M3, E5M2, INT6, INT4) and block size is central to the performance-vs-accuracy tradeoffs:
- **MXFP8 (E4M3)**: matches BF16 in LLM pre-training up to 8B parameters with $\pm0.5\%$ loss, while doubling matmul throughput [2506.08027].
- **MXINT8**: typically outperforms MXFP8 in both accuracy and hardware efficiency at block size 32; enables stable end-to-end training and inference in LLMs [2510.25602].
- **Lower bitwidths (MXFP4, MXINT4)**: 4-bit integer or float quantization incurs significant accuracy degradation unless combined with error mitigation, e.g., Hadamard rotation for INT4, or extra metadata/mantissa bits for MXFP4 [2601.19213][2510.14557].

**Mixed-precision and block-level allocation:** Leading software/hardware co-design systems, such as MASE, MicroMix, MixDiT, allocate higher precision (e.g., MXFP8/MX9) to outlier-rich channels/heads and use MXFP4/6/INT4 for the majority ("energy-based" channel selection, thresholding via quantization error bounds). These approaches spatially mix precisions within layers, achieving $<0.5\%$ accuracy loss at mean bitwidths $<6$ [2508.02343][2504.08398][2307.15517].

**Metadata-enhanced MX formats:** M$^2$XFP [2601.19213] and MX$^+$ [2510.14557] augment standard MX blocks with sub-block or per-block mini-mantissa fields, block max outlier handling, and/or multi-scale metadata. This closes 70–95% of the 4-bit to 8-bit MXFP accuracy gap at only $\sim$0.25–0.5 bits/element storage overhead.


## 4. Hardware Implementations

MXINT formats are extensively supported in co-designed hardware stacks:
- **GPU tensor-cores (NVIDIA Blackwell):** Native MXFP8 hardware-accelerated quantize/dequantize at block boundaries, with 2× the throughput of BF16. Full GEMM workflows quantize both row and column-oriented blocks, accumulating results in FP32 and re-quantizing if downstream ops require MX formats [2506.08027][2508.02343].
- **Custom MAC units (Jack Unit):** Precision-scalable carry-save multipliers, per-block exponent alignment, and sub-word parallelism enable direct execution of integer and FP MX ops on a single datapath, reducing area and power by 1.2–2.0× versus baseline MACs [2507.04772].
- **RISC-V and NPU integration (MXDOTP, SNAX):** MXFP8 and MXINT8 dot products fused into single-cycle instructions, three-stage pipelines, and precision-configurable arrays. Hybrid precision-reduction trees for accumulation allow optimal trade-off between FP32/INT accumulation cost and normalization [2505.13159][2511.06313].
- **FPGA implementations:** Memoryless conversion engines for all OCP-standard MXINT/MXFP variants (E5M2, E4M3, E3M2, E2M3, E2M1, INT8). Full pipeline support for block quantization, accumulation, and re-scaling in custom IP. Open-source hardware and PyTorch libraries (e.g., Brevitas) provide design flexibility for non-standard formats such as INT5, FP6 [2411.03149][2407.01475].


## 5. Empirical Results in Large-Scale Training and Inference

**Language Models (LLMs):**
- **MXFP8-E4M3** achieves BF16-equivalent perplexity and zero/few-shot accuracy on 8B models trained on 15T tokens; throughput is doubled and memory use is halved (per parameter) compared to BF16 [2506.08027].
- **MXINT8** matches or exceeds MXFP8 in both inference (KL divergence, QSNR) and training settings, with hardware savings of 20–40% (area, energy). Loss curves and zero/few-shot accuracy for Llama scale models are effectively indistinguishable from BF16 [2510.25602][2310.10537].
- **MXINT4/NVINT4** with Hadamard rotation outperform corresponding FP formats for certain tasks; without error mitigation, INT4 and FP4 blocks suffer large degradation unless paired with GPTQ or metadata enhancements [2510.25602][2405.07135][2601.19213].
- **Hybrid and metadata-augmented formats:** Mixed-precision allocation (e.g., MicroMix, MixDiT) allows the use of $>$50% MXFP4/MX6/INT4 channels, with selectively higher-precision outlier blocks/channels. M$^2$XFP and MX$^+$ yield sub-1% accuracy drops for average bitwidths near 4.5. MX$^+$ recoups more than 20–40% of the accuracy lost in "bare" 4-bit MX [2510.14557][2601.19213].

**Other modalities:**
- **FFT in MRI:** MXFP8-E4M3 with block size 32, 3-bit mantissa, achieves near-FP16 image quality ($\mathrm{PSNR} \geq 38.9~\mathrm{dB}$, $\mathrm{SSIM} > 0.96$) at $8\times$ compression [2512.04317].
- **Vision Transformers (ViT):** MXINT6/8 yields $<0.1\%$ top-1 loss, $5\times$ memory savings, and $93\times$–$171\times$ speedup relative to Float16 on FPGAs. All operators (incl. Softmax, LayerNorm, GELU) can be mapped to MXINT+LUT structures [2505.22194].
- **Robotics, edge learning:** MXINT8 (and other MX types) enable 51% lower memory use and $4\times$ higher throughput at iso-area/iso-energy compared to prior continuous learning accelerators [2505.22404].


## 6. Limitations, Instabilities, and Mitigations

Though MXINT formats are highly hardware- and accuracy-efficient, there are specific limitations:
- **Training instabilities:** Direct end-to-end LLM training with block quantization of all tensors sometimes exhibits sharp, irrecoverable loss spikes, traced to quantization-induced multiplicative gradient bias—especially in LayerNorm affine parameter and activation blocks where value distributions are tightly clustered [2506.20752].
- **Mitigation:** Retaining higher precision for activations, LayerNorm weights, or backward passes (e.g., quantizing only weights, or forward-only quantization) eliminates instabilities and achieves full convergence [2506.20752].
- **Block size effects:** Larger blocks amortize scale overhead, but yield coarser dynamic-range adaptation; block sizes 32–64 are widely used as a practical compromise [2407.01475].
- **Transpose and channel reordering:** Quantization and transpose are non-commutative; two orientations (row, column) must be stored per weight tensor for GEMM-dominated models [2506.08027].

For ultra-low bitwidth MX (MXFP4, MXINT4), accuracy is only retained if augmented with metadata (M$^2$XFP, MX$^+$), judicious channel/block reordering, outlier-aware allocation, or QAT/PTQ techniques such as GPTQ for weights and SmoothQuant for activations [2601.19213][2508.02343][2405.07135].


## 7. Integration and Best Practices

- **Software support:** Key stacks include NVIDIA Transformer Engine, Megatron-LM, Brevitas (PyTorch-based), and custom pipelined dataflow compilers (MASE). These provide block quantization routines, support arbitrary MXINT/MXFP types, and search for optimal per-tensor precision [2506.08027][2307.15517][2407.01475].
- **Model graph transformations:** Replace GEMM/Conv calls with MXINT/MXFP-aware kernels, keeping master weights in higher precision for training, and quantizing on-the-fly during forward and backward passes [2310.10537][2506.08027].
- **Operational guidelines:** Use E4M3 for all weight/activation/activation-gradient tensors in high-accuracy LLMs; keep embeddings and final projections in BF16 as standard practice. Employ round-to-nearest-even for quantization. For mixed and ultra-low precision, leverage metadata-enhanced formats, outlier-aware selection, and channel reordering [2506.08027][2601.19213].

In sum, Microscaling (MXINT) formats, by combining per-block scaling with flexible low-bit element layouts and supporting rich hardware-software co-design, provide the foundation for aggressive quantization of large models with minimal loss of fidelity, and have rapidly become the industry standard for inference and increasingly for training in AI accelerators [2506.08027][2510.25602][2310.10537][2601.19213][2510.14557].

Source: https://www.emergentmind.com/topics/microscaling-mxint-formats