---
title: Microscaling Floating-Point Formats
url: https://www.emergentmind.com/topics/microscaling-floating-point-formats
type: topic
---

# Microscaling Floating-Point Formats

Microscaling floating-point formats (often abbreviated as MX formats) are block-scaled data representations that share a single floating-point scale among small groups of tensor elements, enabling substantial reductions in storage and compute bandwidth while preserving a large effective dynamic range. The key insight of the microscaling paradigm is to amortize expensive exponent representation over a block, rather than dedicating an exponent to each tensor element as in IEEE floating-point, or using strictly global scales as in quantized integer arithmetic. MX formats have become widely supported in AI hardware, with reference implementations and native acceleration for 4- to 8-bit variants such as MXFP4 and MXFP8 on modern NVIDIA Blackwell Tensor Cores and RISC-V architectures.

## 1. Microscaling Format Structure and Encoding

Microscaling floating-point formats partition tensors into blocks of $B$ elements (typically $B=16$ or $32$), with each block sharing a single scaling factor, usually encoded as an 8-bit floating-point value (E8M0: 8 exponent bits, no mantissa bits). Each element within the block is assigned a narrow sub-word floating-point representation (e.g., S1E2M1 for MXFP4: 1 sign, 2 exponent, 1 mantissa bit). The decoded value for element $i$ in block $j$ is

\[
\hat{x}_i = s_j \cdot m_i
\]
where $s_j$ is the shared scale, and $m_i$ is the decoded mini-float for element $i$ [2606.09686].

Block-scale calculation is performed by selecting $s_j$ so that the largest-magnitude element in the block fits the representable dynamic range:

\[
s_j = 2^{\lfloor \log_2 \max(|X_j|) \rfloor - b}
\]
where $b$ is the exponent bias (fixed by the block-scale representation) [2604.17789].

Within the block, quantization proceeds by normalizing each value by $s_j$, rounding and clipping to fit the mantissa/exponent grid, and storing the resulting code. For element $x_i$,

\[
Q(x_i) = \mathrm{clip}\left( \mathrm{round} \left( \frac{x_i}{s_j} \right), q_{\min}, q_{\max} \right)
\]
with $q_{\min}, q_{\max}$ defined by the element format. Dequantization involves multiplying $Q(x_i)$ by $s_j$ [2604.17789].

## 2. Trade-offs: Block Size, Precision, and Dynamic Range

The microscaling design introduces explicit trade-offs:

- **Block size ($B$):** Smaller $B$ allows finer adaptation to local tensor statistics and better suppression of outliers, but increases metadata overhead (i.e., more shared scales per tensor) and computational cost for GEMM kernels due to more frequent scaling [2604.17789].
- **Element format ($e$, $m$):** Fewer exponent and mantissa bits lower storage/compute cost but increase quantization error, limit representable range, and heighten sensitivity to outliers.
- **Block-scale format:** The block-scale itself is usually quantized (e.g., E8M0, E4M3), introducing quantization noise and range limitations at the block scale as well.

A typical MXFP4 block uses 4 bits/element (S1E2M1) with a single E8M0 (8 bit) scale per 32 elements, yielding 4.25 bits/element average overhead [2606.09686]. Variants such as NVFP4 (NVIDIA) use 16-element blocks with an FP8 E4M3 scale for slightly finer granularity and somewhat higher per-element cost (4.5 bits/element).

The total quantization error in microscaling can be decomposed into element quantization error (discretization of normalized values) and scale quantization/range error. Notably, for extremely small block sizes, discretization of the block scale itself begins to dominate and can paradoxically increase mean-squared error, leading to a "perplexity inversion" where further reductions in $B$ degrade, rather than improve, model quality [2601.19026].

## 3. Outlier Suppression and Format-Aware Transformations

A central challenge of block-scaled formats is the impact of outlier elements within a block. A single outlier elevates the shared block scale, compressing the effective dynamic range allocated to the remaining (usually much smaller) values and causing severe quantization error for these non-outlier elements [2604.17789].

To mitigate this, several strategies have been introduced:

- **Outlier-aware rotations:** DuQuant++ applies a data-dependent block-diagonal rotation ($R$) to activations, designed to spread outlier energy across dimensions within each microscaling group of size $B=32$. This reduces the likelihood that a single coordinate will dominate the shared scale, resulting in significantly lower per-block ${\ell}_2$ quantization errors. The inverse rotation is pre-absorbed into the weights, guaranteeing a lossless transform aside from quantization [2604.17789].
- **Affine preconditioning (SmoothQuant):** Shifts dynamic range between activations and weights via diagonal scaling, partially addressing intensity skew.
- **Fine-grained block rotations (Givens-style):** Additional rotation steps further suppress concentrated outliers, but at increased computational cost.
- **Asymmetric shared scales:** AMXFP4 applies separate shared scales for positive and negative parts of each group, compensating for per-block skewness at minimal hardware cost [2411.09909].
- **Exponent field repurposing (MX+):** MX+ reuses the exponent field of the block-maximum element as extra mantissa bits, enhancing the effective precision of the outlier without additional storage overhead [2510.14557].

Empirically, as demonstrated on LLaMA-3 and other LLMs, these format-aware and data-aware transforms (e.g., DuQuant++, MX+, asymmetric scaling) yield lower quantization error and perplexity compared to data-agnostic rotations or per-channel quantization regimes [2604.17789, 2411.09909, 2510.14557].

## 4. Scale Quantization Effects and Remedies

While decreasing block size typically improves local adaptation, the effectiveness of this strategy is limited by the quantization and dynamic range of the block scale itself. For instance, when using an E4M3 block scale (FP8, 4 exponent + 3 mantissa bits), the smallest nonzero scale is $2^{-9}\approx 0.002$. If tensor blocks are very low-variance (many elements near zero), the quantized scale may round down to zero, causing catastrophic underflow and increased error—a phenomenon confirmed both experimentally and theoretically [2601.19026].

A key remedy is to adopt an expanded scale format. The unsigned FP8 E5M3 ("UE5M3") format, which repurposes the sign bit for an additional exponent bit (exponent 5, mantissa 3, bias 15), extends scale dynamic range down to $2^{-17}$ [2601.19026]. This extension nearly eliminates scale quantization error for narrow-distribution blocks, preventing the error inversion effect and matching the accuracy of global scale compensation but with lower hardware and compute cost.

Scale selection algorithms also matter. Whereas traditional MX selects scales based on per-block maxima, recent approaches such as ScaleSearch choose the scale within quantized scale grid points to minimize overall block quantization error, further reducing error compared to the default approach [2605.12464].

## 5. Post-Training Quantization and Compatibility

Microscaling formats pose unique challenges for post-training quantization (PTQ) relative to integer-based and per-channel floating-point quantization. Empirical studies suggest:

- MXFP8 (E4M3) achieves near-lossless performance for standard PTQ algorithms across weight, activation, and even vision-language modalities, with recovery rates >99% and PPL within 0.2 of baseline [2601.09555].
- MXFP4 (E2M1), despite its hardware efficiency, typically incurs significant accuracy loss unless augmented by format-aware PTQ algorithms such as FlatQuant (learnable affine block shifts), MR-GPTQ (block-wise Hadamard rotations fused into weights), or outlier-aware transforms (DuQuant++, MX+) [2604.17789, 2509.23202, 2510.14557].
- Pre-scale optimization (e.g., multiplying tensors by α ≈ 0.75 before quantization) helps mitigate scale quantization error in MXFP4 pipelines [2601.09555].

Rotational and affine preconditioning paradigms that are effective for integer PTQ may underperform or degrade when directly applied to MXFP4, underscoring the need for format-specialized adaptation [2601.09555, 2509.23202].

## 6. Hardware Implementation and Ecosystem

Microscaling formats have been integrated into multiple hardware pipelines:

- **Compute-in-memory (CIM):** MXFormer demonstrates weight-stationary Transformer acceleration using analog CIM arrays with MXFP4-native arithmetic, achieving <1% accuracy degradation without retraining [2602.12480].
- **RISC-V Extensions:** MXDOTP and VMXDOTP are ISA extensions for RISC-V supporting vectorized and scalar MXFP4/MXFP8 dot-product acceleration with minimal area overhead and ≥97% unit utilization [2505.13159, 2603.04979].
- **NVIDIA Blackwell Tensor Cores and AMD equivalents:** Support block-based MXFP4, NVFP4, MXFP8, and other OCP MX-compliant block floating-point formats natively, exposing these primitives to frameworks [2606.09686].
- **Memory-free floating-to-MXFP converter:** End-to-end combinational hardware for 32-way conversion to all major MX types (E5M2, E4M3, down to E2M1) in a single pipeline stage, using only LUTs, no RAM or DSP blocks [2411.03149].

A comprehensive, vendor-neutral registry of MX and related formats (including conformance vectors and bit-exact test packs) is maintained publicly for engineering compatibility and diagnosis [2606.09686].

## 7. Limitations, Alternative Schemes, and Future Directions

Key limitations of standard microscaling designs include:

- **Sensitivity to outliers:** Despite outlier-aware transforms, MXFP4 and similar low-bit formats can still suffer from severe local error if outliers are untamed [2604.17789].
- **Block-size and scale-inversion tradeoff:** Models with primarily low-variance blocks can see accuracy degradation at extremely fine block sizes due to scale quantization effects [2601.19026].
- **Groupwise asymmetry:** Symmetric block scaling cannot accommodate strong per-block mean-shift or skew, motivating asymmetric shared scales (AMXFP4) and adaptive exponent-mantissa allocation [2411.09909, 2412.19821].
- **PTQ fragility:** Not all quantization algorithms are compatible—rotation-based and naive PTQ can underperform relative to format-aware algorithms [2601.09555, 2509.23202].

Recent extensions and alternatives include:

- **MX+ and MX++:** Exponent field repurposing and finer-grained shared scale structures to capture outlier fidelity without increasing overall storage cost [2510.14557].
- **Nanoscaling (NxFP):** NanoMantissa, adaptive microexponent, and code recycling further compress the footprint while reducing quantization error, especially in sub-6-bit regimes [2412.19821].
- **Static-split alternatives:** GoldenFloat proposes an analytic split rule for static exponent/fraction allocation across bit-widths, with an integer-backed Lucas accumulator, as a cross-width hardware solution. Co-existence of this and other static-split or adaptive formats (e.g., Takum, posit) remains an open investigation [2606.05017].

Consensus in the literature is that aggressive block floating-point scaling with format-aware, data-aware, and hardware-tailored adaption continues to be an active research area, with new directions in block-scale-free architectures (e.g., AetherFloat [2603.08741]), quantization-aware training for sub-8-bit deployment, and joint software-hardware co-design for LLMs and transformer accelerators.

---

**Selected References:**

- DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization [2604.17789]
- Is Finer Better? The Limits of Microscaling Formats in Large Language Models [2601.19026]
- MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving [2510.14557]
- AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference [2411.09909]
- Benchmarking PTQ of LLMs under Microscaling Floating Point Formats [2601.09555]
- An 84-Format Numeric Catalog with Bit-Exact Conformance Vectors [2606.09686]
- Nanoscaling Floating-Point (NxFP) [2412.19821]
- VMXDOTP: A RISC-V Vector ISA Extension for Efficient Microscaling (MX) Format Acceleration [2603.04979]

Source: https://www.emergentmind.com/topics/microscaling-floating-point-formats