Microscaling Quantization Overview

Updated 12 July 2026

Microscaling quantization is a numerical method that groups values in small blocks with one shared scale, improving dynamic-range adaptivity.
It employs low-precision formats like MXINT and MXFP to reduce metadata overhead while preserving local accuracy in matrix operations.
Recent studies emphasize trade-offs in block size, scale encoding, and outlier management, driving innovative PTQ and hardware co-design approaches.

Microscaling quantization is a family of low-precision numerical representations in which a small block of values shares a scale while each element is stored in a narrow integer or floating-point payload. In the Open Compute Project formulation, an MX block represents $k$ values with one shared scale $X$ and per-element codes $\{P_i\}_{i=1}^k$ , with reconstruction $v_i = X P_i$ ; in the concrete formats studied in the literature, $k$ is often 32, though important variants use 16-element, $8\times 8$ , or 128-element groupings (Rouhani et al., 2023). This block-scaled design places microscaling between coarse per-tensor quantization and full per-value floating point: it preserves more local dynamic-range adaptivity than tensor-wise scaling while amortizing exponent metadata across many values, and it has therefore become a central design point for LLM inference and, increasingly, for training-oriented hardware and algorithms (Park et al., 23 May 2026, Cuyckens et al., 28 May 2025).

1. Foundational representation and quantization rules

The canonical MX conversion rule chooses a shared block exponent from the maximum magnitude in the block and then quantizes each normalized element into the target payload format. One explicit formulation computes

$shared\_exp \gets \lfloor \log_2(\max_i |V_i|) \rfloor - emax_{elem}, \qquad X \gets 2^{shared\_exp},$

followed by

$P_i = quantize\_to\_element\_format(V_i / X),$

with clamping of normal numbers that exceed the representable range of the element format (Rouhani et al., 2023). This construction is the basis of both MXINT and MXFP families: MXINT uses integer element payloads with shared scaling metadata, whereas MXFP uses low-bit floating-point payloads with the same blockwise scale-sharing structure (Sharify et al., 2024, Cuyckens et al., 28 May 2025).

Microscaling differs from standard integer PTQ chiefly in the granularity and semantics of scaling. Standard integer quantization is commonly written with one scale and often a zero-point for an entire tensor, channel, or token, while MXINT in the LLM PTQ literature is described structurally as a $d$ -bit integer payload plus an 8-bit shared scale factor per block, with no zero-point (Sharify et al., 2024). In MXFP4, a tensor is partitioned into blocks and each block $j$ uses a shared scale

$X$ 0

so that an element $X$ 1 is quantized by nearest representable FP4 value under that block scale (Lin et al., 20 Apr 2026). NVFP4 introduces a hierarchical variant with a tensor-global FP32 scale $X$ 2 and 16-element FP8-scaled blocks $X$ 3, so the effective dequantization factor becomes $X$ 4 rather than a single E8M0-style block exponent (Bao et al., 12 May 2026).

A practical consequence of this representation is that scaling axes matter. In MX, quantization and matrix transpose do not generally commute, because the shared-scale grouping is attached to a chosen axis or tile layout; this point motivates square or tile-based block designs in training hardware, where both $X$ 5 and $X$ 6 are used repeatedly (Rouhani et al., 2023, Cuyckens et al., 28 May 2025).

2. Numerical formats, block structures, and scale encodings

The literature defines a broad MX design space spanning integer payloads, narrow floating-point payloads, exponent-only shared scales, FP8 block scales, and adaptive dual-mode element encodings. All surveyed formats retain the same basic principle—small groups with shared scale metadata—but differ sharply in block size, scale encoding, and the degree of per-element flexibility (Rouhani et al., 2023, Park et al., 23 May 2026).

Format example	Element payload	Shared-scale scheme / block
MXFP4	FP4 E2M1	E8M0 shared scale, typically block 32
MXFP6 / MXFP8	FP6 E2M3 or E3M2; FP8 E4M3 or E5M2	E8M0 shared scale, typically block 32
NVFP4	FP4 E2M1	FP32 global scale + FP8 E4M3 block scale, block 16
MXSF (MX-SAFE)	FP8 E2M5 or sub-FP E3M2, chosen per element	Shared exponent $X$ 7; inference $X$ 8, training $X$ 9
AMXFP4	FP4 E2M1	Separate positive/negative shared scales

In the original MX data-format study, the concrete OCP-compliant formats were MXINT8, MXFP8, MXFP6, and MXFP4, all using block size 32 and E8M0 as the shared scale format; the evaluated payload variants were FP8 E4M3/E5M2, FP6 E2M3/E3M2, and FP4 E2M1 (Rouhani et al., 2023). Subsequent hardware work retained the same six standardized element types—MXINT8, MXFP8 E5M2, MXFP8 E4M3, MXFP6 E3M2, MXFP6 E2M3, and MXFP4 E2M1—while changing the grouping geometry to $\{P_i\}_{i=1}^k$ 0 square blocks so that the same shared exponent can serve both forward and backward matrix orientations (Cuyckens et al., 28 May 2025).

Several later papers modify the scale or payload allocation rather than the basic MX principle. MXSF repurposes the subnormal region of MXFP8 E2M5 so that values with exponent gap $\{P_i\}_{i=1}^k$ 1 are stored in E2M5 mode, while values with $\{P_i\}_{i=1}^k$ 2 switch to E3M2 with bias $\{P_i\}_{i=1}^k$ 3; its stated purpose is to support both direct-cast inference and full training within the same 8-bit budget (Park et al., 23 May 2026). AMXFP4 keeps FP4 E2M1 elements but replaces the single symmetric shared scale by sign-dependent positive and negative shared scales, using either power-of-two or FP8 E5M2 scale encodings (Lee et al., 2024). NVFP4, by contrast, retains symmetric FP4 elements but uses a two-level scale hierarchy, which later scale-optimization work treats as the main source of avoidable reconstruction error (Bao et al., 12 May 2026).

3. Error sources and distributional phenomena

The dominant error mechanisms in microscaling are more specific than “low precision” in the abstract. A recurring theme is that quantization quality depends jointly on blockwise outliers, the representability of the shared scale itself, and the interaction between block size and within-block statistics (Lin et al., 20 Apr 2026, Zhang et al., 14 Jan 2026).

In MXFP4, activation outliers are particularly harmful because one extreme value can inflate the shared scale for the entire 32-element block. Since the block scale is tied to $\{P_i\}_{i=1}^k$ 4, a single outlier “pulls up” the shared exponent and compresses the effective resolution available to the remaining 31 elements; DuQuant++ measures this using the per-group normalized error

$\{P_i\}_{i=1}^k$ 5

and reports the worst behavior at outlier-heavy down-projection inputs (Lin et al., 20 Apr 2026). A related diagnosis appears in $\{P_i\}_{i=1}^k$ 6: for standard MXFP4, the dominant error source is “misalignment between the block maximum and the coarse shared power-of-two scale,” and preserving the block maximum or refining subgroup scaling with minimal metadata substantially narrows the accuracy gap (Hu et al., 27 Jan 2026).

Scale quantization itself is a first-order error source in MXFP4. A systematic PTQ benchmark under MXFP formats concludes that MXFP8 is consistently near-lossless, while MXFP4 remains difficult largely because E8M0 forces scales to powers of two; replacing low-precision scales by high-precision scales while keeping FP4 values fixed materially improves perplexity, and a simple pre-scale transform $\{P_i\}_{i=1}^k$ 7 mitigates clipping bias under MXFP4 (Zhang et al., 14 Jan 2026). Two later papers make the same point from different angles: SOAR shows that in NVFP4, inflexible and coupled scale selection leaves significant reconstruction error on the table, and ScaleSearch shows that even the standard “max divided by 6” block scale in NVFP4 is frequently not the MSE-minimizing choice (Bao et al., 12 May 2026, Gupta et al., 12 May 2026).

Microscaling also exhibits non-obvious block-size phenomena. “Is Finer Better?” reports that with FP4 elements and FP8 UE4M3 scales, decreasing block size below a model-dependent threshold can worsen perplexity rather than improve it. The paper attributes this inversion to the interplay between narrow tensor distributions and the limited dynamic range of the quantized scales, and it decomposes the total error into non-max-element error, max-element error, and the special case where the block scale rounds to zero (Fasoli et al., 26 Jan 2026). This directly contradicts the common expectation that smaller microscaling groups are always better.

A different but related distributional effect appears in AMXFP4. There, decreasing group size strongly suppresses kurtosis—that is, local outlier severity—but simultaneously increases group-wise asymmetry, measured by nonzero local means. This motivates the paper’s claim that standard symmetric MXFP4 addresses outliers at the cost of increased group-wise asymmetry, and that sign-dependent shared scales are therefore better matched to small-group FP4 inference (Lee et al., 2024).

4. PTQ methods and format-aware mitigation strategies

Microscaling quantization has evolved from a format definition into a family of PTQ, scale-search, metadata, rotation, and mixed-precision methods. The common pattern is format awareness: the best-performing methods do not treat MX as a drop-in replacement for INT quantization, but instead exploit its specific block structure and scale semantics (Zhang et al., 14 Jan 2026, Lin et al., 20 Apr 2026).

The first major adaptation path extended established PTQ methods to MXINT. One LLM PTQ study showed that SmoothQuant can be applied before MX quantization without the additional fixed-point calibration phase, and that GPTQ can be modified to quantize micro-blocks aligned with the MX block size, enabling weight reconstruction under shared-scale quantizers. In that study, MXINT8 largely avoids the INT8 cliff that appears under per-tensor integer quantization, and mixed setups such as activation MXINT8 with weight MXINT4 become viable when paired with SmoothQuant or GPTQ depending on the model family (Sharify et al., 2024).

Later work targeted MXFP4 and NVFP4 more directly. DuQuant++ is explicitly format-aware: it sets the rotation block size to $\{P_i\}_{i=1}^k$ 8 so that each orthogonal transform coincides with one MXFP4 microscaling group, then replaces the original dual-rotation integer pipeline by a single outlier-aware block-diagonal rotation. The simplification relies on MXFP4’s independent per-group scales, which remove the cross-block variance problem that motivated the original zigzag permutation and second rotation (Lin et al., 20 Apr 2026). By contrast, the benchmarking study of MXFP PTQ finds that rotational transformations such as QuaRot and SpinQuant, which are often helpful for INT4, can be worse than RTN under MXFP4, whereas error-compensation methods such as GPTQ and MR-GPTQ and affine methods such as FlatQuant are more compatible with block floating-point quantization (Zhang et al., 14 Jan 2026).

A second mitigation path adds minimal metadata instead of heavier transformations. $\{P_i\}_{i=1}^k$ 9 keeps FP4 data and E8M0 group scaling but adds 8 metadata bits per 32-element group, using element-level extra mantissa for activations and subgroup-level scale refinement for weights; its reported average accuracy loss on 7B/8B models drops to 1.58%, versus 5.38% for MXFP4 and 2.52% for NVFP4, at about 4.5 bits per element (Hu et al., 27 Jan 2026). OPAL takes a different activation-centric route: it preserves the top $v_i = X P_i$ 0 outliers in BF16 for each 128-element activation block, computes the shared scale from the $v_i = X P_i$ 1-th largest exponent, and quantizes the remaining $v_i = X P_i$ 2 values into low-bit integers under that revised scale (Koo et al., 2024). AMXFP4 addresses the asymmetry problem rather than the maximum problem, replacing the single shared scale by positive and negative shared scales and reporting lower MSE than symmetric FP4 variants and better downstream accuracy than standard MXFP4 (Lee et al., 2024).

A third line of work concentrates on scale optimization itself. SOAR derives closed-form updates for NVFP4’s global scale $v_i = X P_i$ 3 and block scales $v_i = X P_i$ 4 under fixed FP4 assignments, then decouples the high-precision quantization scale $v_i = X P_i$ 5 from the FP8 dequantization scale $v_i = X P_i$ 6 through local discrete search (Bao et al., 12 May 2026). ScaleSearch makes a similar point at the block level: in NVFP4, the FP8 E4M3 scale contains mantissa bits, so nearby representable scales can be enumerated by bit-pattern offsets $v_i = X P_i$ 7, and the MSE-minimizing scale is often not the standard max-based choice. The paper reports a 27% reduction in quantization error for NVFP4 and integrates the search into both PTQ and low-precision attention pipelines (Gupta et al., 12 May 2026).

Mixed-precision MX is another major direction. MicroMix partitions activation channels into $v_i = X P_i$ 8, $v_i = X P_i$ 9, and $k$ 0, assigns matching weight channels to MXFP4, MXFP6, or MXFP8 based on thresholded quantization-error estimates, and then executes the resulting partitions with custom Blackwell kernels. This treats microscaling not as a uniform datatype but as a per-layer precision allocation problem over hardware-native FP4/FP6/FP8 paths (Liu et al., 4 Aug 2025).

5. Hardware, kernels, and compiler support

Microscaling quantization is tightly coupled to hardware, because its main purpose is to make native low-bit matrix multiplication practical without sacrificing too much dynamic range. The most visible deployment context in the recent literature is NVIDIA Blackwell, whose Tensor Cores natively support FP4-based microscaling formats and motivate a shift from INT4-centric kernels to MXFP4 and NVFP4 execution paths (Lin et al., 20 Apr 2026, Liu et al., 4 Aug 2025).

At the accelerator level, several papers treat MX as a hardware-software co-design problem rather than just a quantizer. The original MX format paper framed MX as balancing hardware efficiency, model accuracy, and user friction, and showed that MXINT8 can act as a drop-in inference replacement while MXFP6 can support GPT-like training with minimal recipe changes (Rouhani et al., 2023). A compiler line of work then explored mixed-precision MXInt mappings on dataflow accelerators: MASE searches block floating-point mantissa widths jointly with hardware parallelism and dataflow mappings, reporting average precision around 4-bit mantissas with minimal to no accuracy degradation and an average improvement of 24% in $k$ 1 accuracy relative to 8-bit fixed-point designs, with only about 3% overhead in energy efficiency (Cheng et al., 2023).

Training-oriented MX hardware focuses on the arithmetic consequences of shared exponents. One robotics-learning processor supports all six standardized MX formats, uses $k$ 2 square shared-exponent groups to avoid storing both $k$ 3 and $k$ 4, and reports a 25.6% area reduction, a 51% lower memory footprint, and 4x higher effective training throughput than Dacapo under iso-peak-throughput comparison (Cuyckens et al., 28 May 2025). A later MAC-level study argues that the main bottleneck in precision-scalable MX datapaths is accumulation rather than multiplication, proposes a hybrid reduction tree, and reports 657, 1438–1675, and 4065 GOPS/W for MXINT8, MXFP8/6, and MXFP4 respectively in the integrated SNAX system (Cuyckens et al., 9 Nov 2025). MX-SAFE contributes a training-inference accelerator that supports its dual-mode MXSF format and reports 24.9% less total energy consumption than a BF16 baseline while achieving similar accuracy (Park et al., 23 May 2026).

LLM-specific kernels illustrate how microscaling formats interact with modern GPUs. SageAttention3 uses Blackwell FP4 Tensor Cores and NVFP4-style microscaling attention to achieve 1038 TOPS on RTX5090, reported as a 5x speedup over the fastest FlashAttention on that GPU, while requiring a special two-level scaling scheme for the softmax output block $k$ 5 because direct FP4 microscaling of $k$ 6-valued attention probabilities is inaccurate (Zhang et al., 16 May 2025). ScaleSearchAttention builds on NVFP4 attention with better block-scale selection, mixed-precision KV caching, and additional preprocessing, and improves Wikitext-2 perplexity by up to 0.77 points for Llama 3.1 70B relative to the compared FP4 attention baselines (Gupta et al., 12 May 2026). MicroMix extends the same hardware-native idea to full transformer linears, reporting at least 20% faster execution than TensorRT-FP8 on both RTX 5070Ti laptop and RTX 5090, while improving prefill latency and memory efficiency for several Llama and Qwen models (Liu et al., 4 Aug 2025).

Some works remain explicitly concerned with preserving the simplicity of the MX datapath. $k$ 7 adds only lightweight top-1 decode logic, a small correction MAC, and a streaming quantization engine, with reported overheads of 0.26% area and 0.36% power over the baseline accelerator while still delivering up to 1.91 $k$ 8 speedup and 1.75 $k$ 9 energy savings over the compared state-of-the-art MX accelerator (Hu et al., 27 Jan 2026). OPAL likewise routes a small set of preserved activation outliers to BF16 FP units while keeping 96.9% of computations in INT multipliers, reporting 1.6–2.2x energy-efficiency improvement and 2.4–3.1x area reduction with less than 1 perplexity increase (Koo et al., 2024).

6. Training behavior, current status, and open questions

The status of microscaling in training is mixed. The foundational MX data-format study showed that MXFP6 could train GPT-like models from scratch to near parity with FP32 using the same ADAM hyperparameters, and described this as the first demonstration of training generative LLMs at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modification to the training recipe (Rouhani et al., 2023). MX-SAFE then advanced the format-design side of this problem by using adaptive E2M5/E3M2 payload allocation and tile-based blocks; it reports only 0.17% average drop for direct-cast inference and under 0.5% for training relative to BF16/FP behavior in its evaluated workloads (Park et al., 23 May 2026).

At the same time, a large-scale training study of Blackwell-style MX formats reports a more pessimistic phenomenon: across nearly one thousand LLMs trained from scratch, fully low-precision MX training shows sharp, stochastic, unrecoverable instabilities in the loss, especially at larger compute scales. That work proposes a multiplicative-gradient-bias explanation, identifies quantization of layer-norm affine parameters and a small fraction of activations as key triggers, and shows that hybrid configurations—most notably MXFP8 E4M3 weights with BF16 activations, or forward-only quantization—recover performance competitive with full precision (Su et al., 25 Jun 2025). This places an important limit on simplistic interpretations of microscaling as a universally “drop-in” training format.

The current practical status is therefore stratified by precision regime. Under MXFP8, the broad PTQ benchmark finds near-lossless performance across text and multimodal models and recommends W8A8 MXFP as the safest default deployment point (Zhang et al., 14 Jan 2026). Under MXFP4 or NVFP4, by contrast, the format remains highly sensitive to scale selection, outlier handling, and algorithm-format compatibility; both benchmark and method papers agree that 4-bit microscaling is still an active optimization problem rather than a solved replacement for BF16 or FP8 (Zhang et al., 14 Jan 2026, Bao et al., 12 May 2026).

Several open questions recur across the literature. One concerns scale representation itself: E8M0 power-of-two scales are hardware-friendly but coarse, FP8 scales add mantissa precision but still quantize the scale, and UE5M3 has been proposed as a better FP8 scale format for FP4 microscaling because it extends low-end dynamic range without requiring global scaling (Fasoli et al., 26 Jan 2026). A second concerns the correct “unit of preprocessing”: DuQuant++ argues that for MXFP4 the correct unit of outlier management is the microscaling group itself, not a global transform, while AMXFP4 argues that the correct statistical object is the asymmetric micro-group rather than a symmetric block (Lin et al., 20 Apr 2026, Lee et al., 2024). A third concerns how far MX can be pushed in training without selective exceptions. The existing evidence suggests that microscaling increasingly succeeds when numerical format, preprocessing, scale search, accumulation hardware, and kernel implementation are co-designed, rather than optimized independently (Su et al., 25 Jun 2025, Hu et al., 27 Jan 2026, Gupta et al., 12 May 2026).