---
title: Hierarchical Block Floating-Point (BFP)
url: https://www.emergentmind.com/topics/hierarchical-block-floating-point-bfp
type: topic
---

# Hierarchical Block Floating-Point (BFP)

The E6M2 floating-point scale is an 8- or 9-bit floating-point numerical format characterized by 6 exponent bits and 2 mantissa bits, primarily deployed for low-precision training and inference of large language models (LLMs). E6M2 enables aggressive memory and compute reductions while providing a wide dynamic range, making it suitable for both microscaling block-quantization frameworks and static unit-scaling pipelines. Its unique exponent–mantissa tradeoff, scaling law implications, quantization procedures, and empirical performance on transformer architectures have been the subject of recent in-depth arXiv studies.

## 1. Definition and Numerical Properties

E6M2 denotes a floating-point representation with **1 sign bit, 6 exponent bits ($E=6$), and 2 mantissa bits ($M=2$)**, for a total width of 8 (microscaling) or 9 (block-quantized) bits depending on context [2510.01863] [2501.02423]. The exponent bias is $b=2^{E-1}-1=31$, with normalized range for exponent field $k=1\ldots 62$, yielding real exponents $e_{\text{min}}=1-b=-30$ to $e_{\text{max}}=(2^6-2)-b=+31$. The mantissa resolution is $1/4$.

**Dynamic range and precision:**
- **Minimum positive normal:** $2^{-30}$
- **Maximum finite normal:** $(2-2^{-2}) \cdot 2^{31} = 1.75\cdot 2^{31}$
- **Minimum positive subnormal:** $2^{-32}$
- **Unit roundoff (machine epsilon):** $\epsilon \approx 2^{-3} = 1/8$ (relative error $\lesssim 0.125$)

This dynamic range is sufficient to accommodate near-peak intermediate values in LLMs, though the 2-bit mantissa provides only four subdivisions within each power-of-two bin.

## 2. Quantization Methodologies and Scale Computation

### Microscaling Quantization

Microscaling [2510.01863] partitions tensors into blocks (typical size $B=32$ or $128$), sharing a single **power-of-two scale $S$** per block. The scale factor is computed to ensure all values fit within the representable E6M2 range. For a tensor block $X = \{x_1,\ldots, x_N\}$, $S$ is chosen such that:

$$
S = \frac{f_{\text{max}}(E,M)}{\max_i |x_i|}
$$

where $f_{\text{max}}(6,2) = 1.75 \cdot 2^{31}$. Quantized values are $q_i = \operatorname{round}(x_i/S)$; dequantization recovers $\hat x_i = q_i \cdot S$. Blockwise quantization facilitates memory efficiency and hardware alignment.

### $\mu$nit Scaling and Static Scaling

Unit scaling [2502.05967] simplifies the pipeline by statically calibrating $S$ based on empirical peak magnitudes (usually per layer or block). The static $S$ exploits the unit-variance normalization of transformer layers: typical signals then cluster in $|\cdot|\lesssim O(1)$, maximizing dynamic range exploitation. For tensors with atypically narrow dynamic range, a calibrated amplification factor can be folded into $S$ to avoid underutilization of exponent bits, though this must be balanced carefully to avoid outliers saturating to $\pm\infty$.

## 3. Scaling Laws and Performance Theory

Validation loss $L$ for an LLM quantized in E6M2 may be predicted via the unified scaling law [2501.02423]:

$$
L(N, D, E, M, B) =
\frac{n}{N^\alpha} +
\frac{d}{D^\beta} + \epsilon +
\frac{D^\beta}{N^\alpha}
\frac{\log_2 B}{\gamma (E+0.5)^\delta (M+0.5)^\nu}
$$

with empirically fitted constants:
$\alpha=0.2368$, $\beta=0.5162$, $n=69.2343$, $d=6.8973\times 10^4$, $\epsilon=1.9061$, $\gamma=1.1335\times 10^4$, $\delta=3.1926$, $\nu=2.9543$.

Compared to E4M3 (the empirically optimal 8-bit exponent–mantissa split under this framework), E6M2 exhibits slightly higher quantization loss for a similar bit width due to its higher exponent-to-mantissa ratio.

### Exponent vs. Mantissa Impact

Exponent bits have approximately **8% more effect** on loss per bit than mantissa bits ($\delta\approx3.19$ vs $\nu\approx2.95$), thus increasing $E$ is marginally more beneficial for loss minimization given a fixed total bit budget. The optimal bit split for $P=E+M+1$ total bits is:

$$
M_{\rm opt} = \frac{\nu}{\delta+\nu} P - 0.5, \quad
E_{\rm opt} = \frac{\delta}{\delta+\nu} P - 0.5
$$

E6M2 ($E=6$, $M=2$) with $P=9$ approximates $E_{\rm opt}\approx4.2$, $M_{\rm opt}\approx3.8$ for $P=8$; this deviation results in modestly greater quantization error than E4M3 [2501.02423].

## 4. Implementation Details and Numerical Pathologies

### Quantization Granularity

Block-size $B$ selection mediates overhead and quantization error. Empirically, block sizes $B=128$ or channel-wise partitioning yield nearly indistinguishable loss profiles [2501.02423]. Smaller blocks improve scale adaptivity at the cost of greater metadata overhead and scale storage. Exact/integer accumulation within blocks is key to minimizing rounding errors and suppressing in-block overflow.

### Subnormals, Overflow, and Underflow

Values outside the normal exponent range utilize subnormal encoding where possible ($k=0$, $m\ne0$), with underflow truncating to zero and overflow saturating to $\pm\infty$. Static scales or erroneously amplified scales can lead to non-reversible quantization artifacts if data distribution assumptions are violated.

### Gradient Handling and Mixed Precision

Common LLM training schemes retain a full-precision “master copy” of parameters, updating with high-precision optimizers (e.g., AdamW), and re-quantizing for forward/backward propagation in E6M2. Storing gradients in E6M2 is feasible for memory reduction [2510.01863], but dot-products and accumulations should use higher-precision or exact arithmetic to suppress blockwise sum errors.

## 5. Empirical Results and Application to LLMs

Experiments with E6M2 microscaling quantization on GPT-2 (124M parameters, Tiny Shakespeare, $B=32$) demonstrate a **memory reduction of 48% relative to bf16** and 25% to full-precision, with only moderate loss inflation (see table below) [2510.01863]:

| Configuration             | Loss vs. baseline | Rel. error  |
|---------------------------|-------------------|-------------|
| bf16 + master copy        | +0.09%            | +0.0036     |
| Microscaling/float16      | +0.48%            | +0.0187     |
| E4M3 mats/grads + bf16    | +25.7%            | +0.995      |
| Full E4M3 Microscaling    | +122%             | +4.745      |

Inference with E6M2-based (and similar) formats is observed to preserve output coherence, whereas reduction of exponent bits (e.g., E3M4) quickly destabilizes model output due to underflow and overflow artifacts.

## 6. Critical Data Size and Compute-Optimal Precision

Under low-precision quantization, exceeding a critical data size $D_{\rm crit}$ increases loss:

$$
D_{\rm crit} =
\left[
\frac{d\,\gamma\,N^\alpha\,(E+0.5)^\delta\,(M+0.5)^\nu}
{\log_2 B}
\right]^{1/(2\beta)}
$$

For E6M2, $D_{\rm crit}$ is finite but significantly larger than for E4M3 or E2M1 at fixed $N$, meaning practical large-scale LLM training is typically below this threshold [2501.02423].

In compute-constrained training, cost-optimal precision is dictated by total FLOPs ($C$), with 4–8 bits per value found to be optimal across $10^{21}$–$10^{31}$ FLOPs—a range inclusive of E6M2’s 8- or 9-bit width. Using lower bits permits training models of larger width or on more data at fixed compute, while higher bits confer diminishing returns.

## 7. Trade-offs and Practical Recommendations

- **Format trade-off:** E6M2 maximizes dynamic range (large $E$) at the expense of resolution (small $M$), favoring stability for activation and gradient tensors in deep transformers.
- **Scaling methodology:** Static scaling is favored when signal distributions are stable; dynamic or blockwise scaling is preferable for highly non-stationary data [2502.05967].
- **Accumulation fidelity:** Exact accumulators are essential for suppressing blockwise summation errors; in-block oversights can compound quantization loss.
- **Loss minimization:** Exponent bits should be favored for overall model performance given limited bit budget.
- **Application regime:** For practical LLM training and inference, E6M2 offers an attractive compromise of memory efficiency, training stability, and ease of hardware implementation, providing near-optimal scaling loss at high compute efficiency when paired with proper master-copy and mixed-precision techniques [2501.02423] [2510.01863] [2502.05967].

E6M2 floating-point scale, as defined and characterized in current research, serves as a viable precision format for large-scale deep learning under modern quantization and memory-constrained training paradigms.

Source: https://www.emergentmind.com/topics/hierarchical-block-floating-point-bfp