Adaptive Block-Scaled Data Types

• Adaptive block-scaled data types are quantization formats that use fine-grained, data-driven adaptation within small blocks to minimize mean-squared error.
• They dynamically select between floating-point and integer representations per block, optimizing quantization performance and hardware efficiency.
• Empirical evaluations show that formats like IF4 reduce MSE and enhance transformer training and inference with minimal memory overhead.

Adaptive Block-Scaled Data Types are quantization formats for low-precision deep learning and scientific computation that utilize fine-grained, data-driven adaptation of numerical representation within small data blocks. These data types enable each group of values—typically size 16—to select locally optimal quantization schemes, notably choosing between floating-point and integer discretizations, in order to minimize mean-squared quantization error. Recent innovations in this class, such as the IF4 (Int/Float 4) format, demonstrate improved accuracy and hardware efficiency by exploiting block-wise adaptivity without extra memory overhead. Adaptive block-scaled designs extend and generalize established approaches like block floating point (BFP) and are supported by both theoretical error analyses and experimental results across transformer training, inference, and custom accelerators (Cook et al., 30 Mar 2026, Soloveychik et al., 2022, Rouhani et al., 2023).

1. Conceptual Foundations: Block-Scaled and Adaptive Formats

Block-scaled data types partition tensors into small blocks—typically 16 elements—and represent each block using low-precision mantissas and a shared scale factor. Standard block floating point (BFP) quantizes all mantissas to a uniform grid, scaling all values in a block by a power-of-two factor. Scaled BFP (SBFP) increases dynamic range by encoding the shared scale at higher precision (Soloveychik et al., 2022). These approaches permit efficient hardware realization of inner products with high accuracy and energy efficiency.

Adaptive block-scaled data types generalize this principle by allowing each block to dynamically select a quantization format from a set of choices, such as floating-point (with nonuniform level spacing) or integer (uniform grid), depending on which achieves lower quantization error in that block (Cook et al., 30 Mar 2026). This block-level adaptivity targets the heterogeneity of real-valued data distributions, such as outlier prevalence or local uniformity, improving per-block and overall quantization fidelity.

2. Technical Definition and Realizations

In IF4, each block of 16 values from a tensor $X^{(i)} \in \mathbb{R}^{16}$ is scaled by a shared FP8 (E4M3) scale $\Delta_i$. Each value $x_j$ in the block is then normalized and quantized in two parallel paths:

• FP4 path:

$u_j = x_j / \Delta_i$

$$\bar{u}_j^{(\mathrm{FP})} = \mathrm{fp4\_e2m1}(u_j)$$

$$\hat{x}_j^{(\mathrm{FP})} = \Delta_i \bar{u}_j^{(\mathrm{FP})}$$

• INT4 path (6/7 trick):

$v_j = u_j \cdot \frac{7}{6}$

$\bar{u}_j^{(\mathrm{INT})} = \mathrm{int4}(v_j)$

$$\hat{x}_j^{(\mathrm{INT})} = \Delta_i \cdot \left( \bar{u}_j^{(\mathrm{INT})} \cdot \frac{6}{7} \right)$$

For each block, mean-squared reconstruction errors ($E_{\mathrm{FP}}$, $\Delta_i$0) are computed for both representations. The representation with the lower $\Delta_i$1 is chosen for that block. This choice is encoded by repurposing the unused sign bit of the block scale factor.

Generalizations to other bit-widths include IF3 (mixing 3-bit float and int) and IF6 (mixing 6-bit float and int, with multiple exponent–mantissa configurations). For each, the adaptive selection is based on minimizing block-wise quantization error (Cook et al., 30 Mar 2026).

3. Theoretical Error Analysis and Performance Bounds

Traditional block floating point error is dominated by the combination of quantization granularity and scale-factor alignment within each block. For SBFP and BFP, rigorous bounds are provided for the inner product error between independently quantized normally distributed blocks. Variance expressions and sub-Gaussian tail bounds are derived for both formats, with BFP error exhibiting “jumps” at block sizes where the expected maximal input crosses power-of-two thresholds (Soloveychik et al., 2022):

• SBFP inner-product error variance:

$\Delta_i$2

• BFP inner-product error variance (for $\Delta_i$3):

$\Delta_i$4

The adaptive IF4 achieves lower mean-squared quantization error by block-wise minimization, reducing MSE on $\Delta_i$5 from $\Delta_i$6 (NVFP4) to $\Delta_i$7 at 4 bits, corresponding to significant improvements in effective signal-to-quantization-noise ratio. Performance is further characterized via the REBAC ($\Delta_i$8) accuracy metric (Soloveychik et al., 2022).

4. Comparison with Alternative Adaptive Approaches

Alternative adaptive quantization schemes employ additional forms of scale sharing and adaptivity. The Block Data Representation (BDR) framework generalizes adaptive block-scaled design by introducing a hierarchical structure with two levels of scaling: a coarse global exponent and finer-grained “microexponents” assigned to sub-blocks (e.g., pairs of values) (Rouhani et al., 2023). The MX format (e.g., MX6, MX9) thus encodes per-block exponents and per-sub-block microexponents, providing ultra-fine scale adaptation.

Empirical results indicate that formats such as MX9 (9 mantissa bits, 5 global, 1 micro exponent) deliver FP32-level accuracy in training and inference, while MX6 (6 mantissa) delivers $\Delta_i$9 memory/area efficiency over FP8 at similar QSNR. For low-bit adaptive block-scaled types such as IF4, block-level adaptivity via float/int selection attains error minimization with lower hardware complexity (no sub-block exponents) and zero additional storage.

5. Empirical Evaluation in Training, Inference, and Hardware

Empirical evaluation demonstrates the superiority of adaptive block-scaled data types on transformer training, post-training quantization (PTQ), and hardware metrics (Cook et al., 30 Mar 2026):

• Quantized Training: In W4A4G4 transformer pretraining (340M parameters), IF4 yields lower cross-entropy loss than NVFP4, with additional gains using unbiased backward transforms.
• Inference and PTQ: Across LLMs (Qwen 3.5, Nemotron 3, 4B–122B), IF4 achieves lower perplexity than NVFP4 and NVINT4, reducing the average PTQ gap to BF16. Zero-shot accuracies on ARC, HellaSwag, LAMBADA, and PIQA show IF4 closing approximately half of the BF16–NVFP4 accuracy gap.
• Hardware Synthesis: IF4 MAC units synthesized in TSMC 28nm process exhibit minor overhead versus NVFP4 (latency +4.7%, area +67%, energy efficiency 1111 GFLOPS/W vs 1488 GFLOPS/W), which become negligible at the accelerator level due to memory-bound operation.

A summary of comparative results:

Format	Perplexity (Qwen 122B)	Downstream Accuracy (%)	Energy Eff. (GFLOPS/W)
BF16	5.72	81.5	—
NVFP4	6.20	80.9	1488
NVINT4	6.28	80.9	—
IF4	6.10	80.9	1111

6. Implementation Details and Extensions

The adaptive block-scaled paradigm is directly realizable in hardware. The IF4 MAC architecture decodes each block using the scale’s sign bit to route operand fields through FP4 or INT4 dequantization units (LUT for FP4, shift/adder for INT4), multiplies the 16 operand pairs, applies the combined block scale, and accumulates the results (Cook et al., 30 Mar 2026). The approach can be generalized to other bit-widths (e.g., IF3, IF6) and extends to further hardware/algorithmic combinations.

Block size of 16 is motivated by alignment with tensor-core operations on current GPUs, however, smaller or larger block sizes remain an open area for exploration, with theoretical guidance available for optimal $x_j$0 given bit-width and error targets (Soloveychik et al., 2022). For mixed-precision training or deployment, adaptive formats could, in principle, select among multiple bit-widths or dynamically vary block sizes.

7. Limitations, Open Questions, and Future Directions

Despite strong empirical and theoretical performance, certain limitations are acknowledged. Stochastic rounding with a per-block adaptive selection introduces slight bias, especially affecting extreme values, though observed impacts are negligible. The fixed block size (e.g., 16) is a hardware and deployment-driven choice; investigating adaptivity in block size or per-block bit-width allocation is a prospective research path.

Full integration into major frameworks and accelerators, as well as extension to dynamic precision or scaling hierarchies (as in the MX family), represent further opportunities for increasing hardware and algorithmic efficiency while maintaining model accuracy. The convergence of block-level adaptivity, analytic error bounding, and hardware-realizable designs delineate adaptive block-scaled data types as a central tool in scalable deep learning model deployment (Cook et al., 30 Mar 2026, Soloveychik et al., 2022, Rouhani et al., 2023).