Block Floating-Point (BFP) Format

• Block Floating-Point (BFP) Format is a numerical representation where a group of values shares a common exponent with individual fixed-precision mantissas to manage quantization error and hardware efficiency.
• It systematically balances dynamic range and fixed-point scaling, enabling optimized neural network computations, memory-bound scientific computing, and low-power DSP implementations.
• Recent advancements include adaptive scaling, hierarchical exponent strategies, and accelerator co-designs that enhance performance and energy efficiency while mitigating quantization challenges.

Block Floating-Point (BFP) Format defines a class of numerical representations in which a block of values shares a single exponent ("block exponent"), while each element retains its own fixed-precision mantissa (possibly signed). This approach systematically interpolates between floating-point’s per-element dynamic range and fixed-point’s per-block scaling, with the goal of maximizing hardware efficiency while controlling quantization error. BFP is central to modern hardware-efficient neural network training and inference, memory-bound scientific computing, and low-power embedded DSP systems. Recent research has advanced both the theoretical properties and hardware realizations of BFP, including new adaptive and hierarchical scaling schemes, precise error analyses, and system-level accelerator co-designs (Wang et al., 4 Feb 2026, Soloveychik et al., 2022, Rouhani et al., 2023, Cook et al., 30 Mar 2026, Noh et al., 2022, Song et al., 2017, Han et al., 22 Apr 2025).

1. Mathematical Structure and Encoding

The block floating-point representation encodes a vector or tensor block $x = (x_1, \ldots, x_N)$ as:

• Shared exponent: $E$, selected (usually) as the maximum of the constituent floating-point exponents $e_i$:

$E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$

for group $\mathcal{G}$ (block) of size $N$.

• Per-element mantissas: Each element is quantized to a fixed-width integer mantissa $M_i$ (usually $b$ bits), with alignment determined by $E$.

$$M_i = \left\lfloor \frac{m_i \times 2^{e_i - E + (b-1)}} {1} \right\rfloor$$

$E$0

where $E$1 is the sign.

Canonical BFP block storage thus consists of:

• $E$2 mantissas, $E$3 bits each (signed or sign-magnitude)
• one shared exponent per block (bitwidth determined by target dynamic range)
• Optional: block-level sign or explicit metadata for special modes

Variants:

• "Scaled BFP" (SBFP) stores the per-block scale $E$4 (rather than a power-of-two exponent) in full or wider precision for minimal error (Soloveychik et al., 2022).
• Hierarchical BFP and block data representation schemes introduce secondary exponent fields for sub-blocks to capture local statistics (e.g., shared microexponents, as in MX/HiFloat4) (Luo et al., 11 Feb 2026, Rouhani et al., 2023).

2. Block Size, Bitwidth, and Precision/Accuracy Trade-offs

Block Size

• Larger blocks: Amortize exponent overhead, reduce metadata per value, but increase the probability of large intra-block dynamic range, raising the risk of over-shifting small values and thus error on low-magnitude elements (Song et al., 2017, Soloveychik et al., 2022, Xu et al., 2024, Rouhani et al., 2023).
• Smaller blocks: Offer finer-grained local scaling, reducing quantization error for moderate values, at the cost of increased exponent storage/bandwidth.

Empirical and theoretical results:

• For DNN weights/activations, block sizes of 16–64 are common (Wang et al., 4 Feb 2026, Haris et al., 15 Oct 2025, Noh et al., 2022).
• Theoretical optimum for 4-bit mantissa gives $E$5 for minimum relative inner-product error (REBAC) (Soloveychik et al., 2022).
• HiFloat4 demonstrates benefits for group size 64 combined with multi-level microexponents (Luo et al., 11 Feb 2026).

Mantissa Bitwidth

• Wider mantissas: Reduce quantization noise. Empirically, 8 bits (including sign) yields $E$60.3% model accuracy loss on ImageNet-class CNNs, while 4–6 bits are sufficient for small models (Song et al., 2017, Noh et al., 2022, Zhang et al., 2021).
• Narrow mantissas: Enable aggressive memory/compute reduction but can sharply increase accuracy loss unless mitigated by local scaling schemes (adaptive/overlap/microexponents).

Format	Optimal block size	Typical mantissa width	Accuracy drop (typical)
Plain BFP	16–64	4–8 bits	0.1–1.5%
SBFP	$E$7BFP, more bits	$E$8BFP	lower, saturating
HiFloat4	64	4 bits S1P2	$E$91%
IF4/NVFP4	16	4 bits	$e_i$02% (w/ adaptation)

3. Algorithmic Implementation and Adaptive Extensions

Standard Workflow

1. Block formation: Partition tensor into contiguous or strided blocks of a given size.
2. Exponent selection: Max exponent of the block (or local median/pivot in adaptive methods).
3. Mantissa alignment: Right-shift (or occasionally, left-shift) each mantissa so all are scaled to the shared exponent; round to target precision.
4. Block encoding: Store per-block exponent and per-element mantissas. Optionally, additional metadata (as in BBFP’s overlap flags, or IF4’s sign-bit encoding).

Adaptive and Hierarchical Schemes

• Asymmetric/bit-aware allocation: Select mantissa width dynamically within a layer (e.g., full width for attended tokens, reduced for the rest in LLM KV caches) (Wang et al., 4 Feb 2026).
• Outlier smoothing: Weight-space calibration plus runtime channel offset minimization to suppress blockwise outliers in attention matrices (Wang et al., 4 Feb 2026).
• Block-adaptive format selection: At the per-block level, select between alternative quantization forms (e.g., FP4 vs INT4) by MSE minimization (as in IF4) (Cook et al., 30 Mar 2026).
• Hierarchical exponent schemes: HiFloat4 (HiF4) stores a global FP8 scale, eight 1-bit exponents for 8-value groups, and sixteen 1-bit exponents for 4-value subgroups, providing both wide dynamic range and local adaptation (Luo et al., 11 Feb 2026).
• Bidirectional BFP (BBFP): Each element gets a flag and overlap bits, allowing both left- and right-shifts for mantissas, reducing the probability of catastrophic over-shrinking for moderate values (Han et al., 22 Apr 2025).
• Pivot-focus and adaptive grouping (DBFP): Choose central exponents (median or soft cluster centroids) per block to minimize mean absolute deviation, especially for nonlinear ops (e.g., softmax) (Wang et al., 21 Jan 2025).

4. Hardware Realization, Compute Flow, and Accelerator Design

Fixed-Point MAC Arrays

The most salient benefit of BFP is the reduction of all intra-block arithmetic to fixed-point integer MACs, with only one (block-level) scaling. Hardware datapaths can thus operate at much higher throughput and lower area versus IEEE-754 floating-point (Drumond et al., 2018, Noh et al., 2022). Design details include:

• Exponent/max finding: Blockwise comparator trees (e.g., 32-input for $e_i$1 (Wang et al., 4 Feb 2026)) or hierarchical decoders handle exponent extraction.
• Per-block pipeline: Mantissa alignment using parallel shift units; rounding logic for quantization (deterministic or stochastic).
• Configurable MAC arrays: FlexBlock, F-BFQ, and BBAL allow multiple block precisions, dynamic switching among BFP variants, and extending BFP arithmetic to nonlinear functions via dedicated LUT units (Noh et al., 2022, Haris et al., 15 Oct 2025, Han et al., 22 Apr 2025, Wang et al., 21 Jan 2025).

Accelerator	Supported BFP modes	Notable features	Throughput/area benefit
Harmonia (Wang et al., 4 Feb 2026)	M8W4, M8M4, M8M8 (activ/weight bitwidths)	All-layer BFP, reconfigurable PE array, pipelined FP16-BFP converter, tiling-aware dataflow	$e_i$2–$e_i$3 throughput/area
FlexBlock (Noh et al., 2022)	FB12, FB16, FB24	Hierarchical MAC mapping, per-tensor adaptive precision	$e_i$4–$e_i$5 speed, $e_i$6–$e_i$7 energy
F-BFQ (Haris et al., 15 Oct 2025)	Q2 (2-bit), Q3 (3-bit)	Two BFP variants per layer, dynamic switching	$e_i$8 (vs. ARM-CPU/ref)
BBAL (Han et al., 22 Apr 2025)	BFP, BBFP (4,6 bits)	Bidirectional alignment, LUT-based nonlinear PE	$e_i$9 accuracy / $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$0 area gain (vs. outlier-aware INT8)

Nonlinear/Softmax Inference

Recent BFP research targets all-activation BFP for attention/nonlinear layers (previously, BFP was confined to linear ops due to accuracy loss in softmax, SiLU, etc.):

• DBFP + DH-LUT: Pivot-focused exponent alignment plus 2D hierarchical LUT for exp/softmax; avoids intermediate FP; achieves $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$10.1% accuracy loss and $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$2 throughput/latency gains in softmax (Wang et al., 21 Jan 2025).
• BBFP: Overlap bits and bidirectional alignment allow even nonlinear PEs (e.g., softmax LUT) to be implemented via simple table lookup and shifter-multiplier logic (Han et al., 22 Apr 2025).

5. Error Analysis and Theoretical Foundations

BFP quantization error arises primarily from mantissa truncation after shared log2 exponent alignment. Precise characterization is essential for both architecture and model design.

Inner-Product Error Bounds

• Asymptotics: For $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$3-dimensional vectors, mantissa $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$4 bits, error variance $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$5 in SBFP and similar, with BFP incurring jumps at block sizes where the block scale crosses a power-of-two threshold (Soloveychik et al., 2022).
• Empirical: 8-bit mantissas suffice for CNNs on ImageNet with $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$60.3% drop, even with large blocks (Song et al., 2017).
• REBAC ratio: Ratio of BFP/SBFP variances, minimized at block $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$7 for $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$8-bit mantissa (Soloveychik et al., 2022).
• Hierarchical/microexponent schemes: MX6 (6 mantissa bits, 5+1 exponent bits over 16+2 elements) achieves up to $E = \max_{i \in \mathcal{G}}\bigl(e_i\bigr)$9 dB QSNR improvement at half the cost of plain BFP16 (Rouhani et al., 2023).
• Local quantization error (HiFloat4): MSE(HiF4):MSE(NVFP4):MSE(MXFP4) $\mathcal{G}$0 under i.i.d. Gaussian (Luo et al., 11 Feb 2026).

6. Applications and Systemic Impact

Deep Learning Training and Inference

• Tuples of block size / mantissa width can be customized per layer (BitQ) to optimize energy, bandwidth, and accuracy under on-chip constraints (Xu et al., 2024).
• All-layer BFP with hybrid optimization (asymmetric bit allocation, outlier smoothing) enables up to $\mathcal{G}$1 speedup, $\mathcal{G}$2 energy efficiency over FP16 baselines across multiple LLMs (Harmonia) (Wang et al., 4 Feb 2026).
• Bidirectional and adaptively block-scaled BFP (BBFP, IF4, HiFloat4) close the gap to full precision for aggressive quantization (4 bits/weight and below), mitigating quantization spikes and outlier-induced loss (Han et al., 22 Apr 2025, Cook et al., 30 Mar 2026, Luo et al., 11 Feb 2026).

Scientific Computing and Embedded DSP

• BLAS and Multigrid: BFP enables energy-efficient all-integer routines for matrix-matrix/vector operations with explicit quantization driver routines for precision management (Kohl et al., 2023).
• Communication DSP: Complex BFP with explicit box encoding achieves near-float EVM in QAM transceivers with ~40% wordlength savings for mantissa widths of 10–12 bits (Choo et al., 2017).
• ReRAM in-memory compute: ReFloat’s block local pivoting exponent encoding maps high-precision MVMs onto bit-serial crossbar hardware, saving order-of-magnitude cycle time vs. FP64 (Song et al., 2020).

7. Limitations, Controversies, and Research Directions

• Block-size tuning is nontrivial: Oversized blocks can induce catastrophic quantization via outlier domination, whereas undersized blocks inflate metadata (Soloveychik et al., 2022, Song et al., 2017, Rouhani et al., 2023).
• Nonlinear operation quantization: Conventional BFP fails under softmax or wide-magnitude nonlinearities (Wang et al., 21 Jan 2025, Han et al., 22 Apr 2025); adaptive exponent sharing and LUT-based acceleration are emerging as standard solutions.
• Hierarchical and hybrid scaling: Microexponent and bidirectional schemes (MX, HiFloat4, BBFP) improve accuracy at modest cost but add minor hardware complexity. Their Pareto relationship to simple BFP is a recent focus (Luo et al., 11 Feb 2026, Rouhani et al., 2023).
• Error distribution: Block-based quantization error is not uniform. Adaptive schemes (e.g., IF4, BBFP) use per-block selection or outlier shifting; selecting between INT and FP quantizers for each block demonstrably minimizes worst-case error (Cook et al., 30 Mar 2026, Han et al., 22 Apr 2025).
• Training vs. Inference: Many BFP benefits accrue only under Quantization-Aware Training (QAT); naive Post-Training Quantization (PTQ) can bottleneck accuracy in single-precision-free BFP variants (Morisaki, 26 Feb 2026).

---

References:

(Wang et al., 4 Feb 2026, Soloveychik et al., 2022, Rouhani et al., 2023, Cook et al., 30 Mar 2026, Noh et al., 2022, Zhang et al., 2021, Song et al., 2017, Luo et al., 11 Feb 2026, Haris et al., 15 Oct 2025, Song et al., 2020, Kohl et al., 2023, Choo et al., 2017, Han et al., 22 Apr 2025, Wang et al., 21 Jan 2025, Xu et al., 2024, Drumond et al., 2018)