BOF4-S: Signed Absolute Block Maximum Norm

• Signed Absolute Block Maximum Normalization (BOF4-S) is a block-wise quantization technique that preserves the sign of the maximum weight in each block to minimize quantization error.
• It optimizes codebook centroids using analytic and data-driven Lloyd-style iterations, achieving lower MSE and MAE than previous methods like NF4, AF4, and BOF4.
• By integrating with outlier-preserving quantization, BOF4-S ensures efficient 4-bit compression and improved downstream performance on benchmarks such as WikiText-2.

Signed Absolute Block Maximum Normalization (BOF4-S) is a block-wise normalization and quantization method that improves the memory efficiency and accuracy of LLM compression by precisely maintaining the signed maximum within weight blocks and optimizing quantizer codebooks accordingly. Introduced as a central component of the BOF4 quantizer family, BOF4-S leverages signed absmax normalization to reduce quantization error beyond prior methods such as NF4, AF4, and the original BOF4, enabling less degradation in both quantization error metrics and downstream LLM performance (Blumenberg et al., 10 May 2025).

1. Formal Definition

Given full-precision weights partitioned into disjoint blocks $\{w_{b,1},\dots,w_{b,I}\}$ for $b=1,\dots,B$ (block size $I$), BOF4-S normalization departs from conventional (unsigned) absmax normalization by preserving the sign of the largest-magnitude weight in each block. Specifically, the signed block maximum for block $b$ is

$$w_b^{\max} = w_{b,j^*} \quad \text{where} \quad j^* = \arg\max_{1\le i\le I} |w_{b,i}|.$$

Each weight is normalized as

$$x_{b,i} = \frac{w_{b,i}}{w_b^{\max}} \quad \in [-1, 1],$$

where $x_{b,j^*}=+1$ and $x_{b,i}$ achieves $-1$ only in degenerate cases. The normalized vector is then quantized using a fixed, block-independent scalar quantizer $$\tilde Q: [-1, 1] \to \{\hat x(1),\dots,\hat x(16)\}$$:

$b=1,\dots,B$0

Decoding reverses this process by reconstructing weights via the codebook with the block’s signed maximum.

2. Mathematical Distinction from Unsigned Normalization

Standard absmax normalization places CDF mass at both $b=1,\dots,B$1 and $b=1,\dots,B$2, specifically $b=1,\dots,B$3 at each, with the remainder distributed over $b=1,\dots,B$4. In contrast, BOF4-S (signed) normalization collapses all such “edge” mass onto $b=1,\dots,B$5 only, removing mass from $b=1,\dots,B$6. Explicitly, using $b=1,\dots,B$7 for the continuous part,

• Unsigned:

$b=1,\dots,B$8

• Signed:

$b=1,\dots,B$9

This change eliminates the requirement for a quantization level at $I$0, releasing a degree of freedom in the codebook and lowering average distortion.

3. Quantization Error Reduction Mechanism

The improvement in error metrics arises because BOF4-S exploits the revised normalized distribution to optimize codebook locations. All methods (NF4, AF4, BOF4, BOF4-S) enforce exact representation of zero and block maximum, but differ in the remaining level placement:

• NF4 uses a Gaussian-quantile heuristic, not true distortion minimization, with unequal level usage.
• AF4 minimizes the MAE of normalized weights presuming Gaussianity but omits scaling, thus not end-to-end optimal.
• BOF4 minimizes MSE (or MAE) of $I$1, under unsigned normalization.
• BOF4-S minimizes the same objectives but over the signed-normalized distribution, shifting centroids due to altered edge mass, yielding strictly lower MSE and MAE under the same Gaussian assumptions.

Empirical results using Gaussian-weighted simulations show BOF4-S outperforms alternatives in both mean absolute and mean squared error at all tested block sizes, and achieves lower language modeling perplexity on benchmarks such as WikiText-2 and LAMBADA, outperforming NF4, AF4, and regular BOF4 (Blumenberg et al., 10 May 2025).

4. Optimization of Codebook Parameters: Analytic and Data-Driven Methods

BOF4-S codebooks can be obtained:

• Analytically (Lloyd/EM):

One adapts the Lloyd algorithm to minimize block-quantization error, with reconstruction level updates:

$I$2

where $I$3 is the PDF of absolute maxima, $I$4 is the CDF conditioned on the block maximum, and $I$5 are decision boundaries. For MAE, use weighted medians [(Blumenberg et al., 10 May 2025) Eqns. (15),(21)].

• Monte Carlo/Empirical:

Draw a large sample $I$6, compute normalized $I$7, assign regions, then update centroids by weighted means (for MSE) or weighted medians (for MAE) with weights $I$8. Iterate until convergence. Empirical codebooks closely match the analytic solution (−56 dB MSE difference), confirming practicality [(Blumenberg et al., 10 May 2025), Table 11].

5. Empirical Performance Results

BOF4-S has been extensively benchmarked both in simulation and on real LLMs. Salient outcomes reported in the source material include:

• Quantization Error (Fig. 3): Across all block sizes $I$9, BOF4-S achieves lower MAE and MSE than AF4, NF4, and the original BOF4, with a 5–10% reduction relative to BOF4.
• Downstream Task Perplexity (Table 1): On Llama-3.1 8B, Qwen-2.5 7B, and Mistral 7B at $b$0, BOF4-S (with OPQ) attains the lowest MAE, MSE, and perplexity on WikiText-2 among 4-bit quantization methods (e.g., 8.43 BOF4-S vs. 8.53 for NF4 and 8.51 for AF4 on Llama).
• Block Size Scaling (Fig. 4): As block size increases up to $b$1, BOF4-S (MSE) consistently yields lower perplexity than NF4 and AF4, with further improvements from OPQ.
• Inference Accuracy (Table 2): On smaller models (Llama-3.2 3B, Qwen-2.5 3B), BOF4-S+OPQ leads or equals the best normalized average accuracy across MMLU, ARC-Challenge, HellaSwag, etc.
• Quantized Fine-tuning with QLoRA (Tables 3–4): BOF4-S (MSE) with OPQ matches or surpasses the BF16 baseline in instruction- and code-specialized fine-tuning of Llama-3.2 3B, and outperforms NF4/AF4 on task metrics.

6. Algorithmic Implementation

The codebook for BOF4-S (MSE) is designed via a data-driven Lloyd-style iteration as follows:

$b$3

At inference (decode) time, for each block, compute the signed maximum, normalize, quantize via the codebook, and reconstruct by product with the signed maximum. Combined with OPQ (storing outlier weights at 16 bits), BOF4-S achieves state-of-the-art 4-bit quantization performance for LLMs (Blumenberg et al., 10 May 2025).

7. Practical Implications and Placement Within Block-wise Quantization

By releasing the restrictive requirement for symmetric edge-levels at $b$2, BOF4-S enables a more efficient partition of quantization levels, which directly lowers end-to-end quantization distortion in both synthetic (Gaussian) and empirical LLM weight distributions. BOF4-S’s design is compatible with outlier-preserving quantization (OPQ), which stores extreme weights at higher precision, addressing the long-tailed nature of empirical weight distributions and further improving perplexity robustness as block size increases. BOF4-S thus constitutes the current standard for 4-bit block-wise quantization precision and LLM applicability, as evidenced by superior performance across major open-source LLM benchmarks (Blumenberg et al., 10 May 2025).