---
title: 'NF4: 4-bit NormalFloat in Neural Quantization'
url: https://www.emergentmind.com/topics/4-bit-normalfloat-nf4
type: topic
---

# NF4: 4-bit NormalFloat in Neural Quantization

A 4-bit NormalFloat (NF4) is a low-bit quantization format specifically constructed for efficient neural-network weight representation, particularly in large language models (LLMs). NF4 employs a 16-level codebook based on the quantiles of a standard normal distribution and utilizes a blockwise absolute-maximum (absmax) normalization strategy. The format was introduced in the context of QLoRA, enabling 4-bit quantized finetuning of models up to 65B parameters on a single GPU, and has since become foundational in block-quantization pipelines for both inference and finetuning. Ongoing research critiques its theoretical optimality and offers data-driven variants that further minimize quantization error.

## 1. NF4 Encoding, Codebook Construction, and Blockwise Quantization

NF4 represents each real-valued parameter $w_{i}$ in a contiguous block of size $B$ using 4 bits, mapping to a set of 16 codepoints $q_j$ shared across all blocks. The standard quantization pipeline comprises:

- **Blockwise scale computation**: $M = \max_{i}|w_i|$
- **Normalization**: $x_i = w_i / M \in [-1,1]$
- **Assignment**: $c_i = \arg\min_j|q_j - x_i|$
- **Storage**: 4-bit index $c_i$ per entry, plus $M$ per block
- **Dequantization**: $\hat{w}_i = M \cdot q_{c_i}$

NF4 codebook centroids $\{q_j\}$ are precomputed by partitioning the standard normal distribution $N(0,1)$ into 16 equiprobable intervals. The endpoints $q_0 = -1.0$ and $q_{15} = +1.0$ are fixed to ensure exact representation of the block maxima. The remaining quantiles are calculated as averages of adjacent cumulative-probability intervals, then renormalized to $[-1,1]$ [2305.14314].

Below is a schematic of the NF4 bit layout and codebook properties:

| Bits         | Description               | Representative Values      |
|--------------|--------------------------|----------------------------|
| 1            | Sign                     | $\pm$                      |
| 2            | Exponent (with bias)     | -1, 0, 1, 2                |
| 1            | Mantissa                 | 1.0, 1.5 (or subnormal 0)  |

A special codeword (exp=0, mant=0, sign=0) is reserved for the exact representation of zero [2505.06653].

## 2. Information-Theoretic Motivation and Block Size Dependence

NF4 was motivated by the argument that, for i.i.d. $N(0,\sigma^2)$ weights, partitioning the real line into equal-probability intervals (using normal quantiles) would maximize entropy and minimize MSE. However, absmax block normalization changes the effective distribution of values to be quantized; normalized weights $X_i = Z_i / M$ are no longer i.i.d. Gaussian but are instead density-concentrated around zero as $B$ increases. This means the original "information-theoretic optimality" argument for the NF4 codebook does not precisely hold outside the scalar case [2306.06965].

An alternative is the $L_1$-optimal "AF4–B" codebook, which computes codepoints by minimizing the absolute reconstruction error for the actual normalized distribution within a block of size $B$. At small $B$ (e.g., $B=64$), both codebooks perform nearly identically, while for very large $B$ the $L_1$-optimized variant is measurably superior [2306.06965].

| Block Size  | Optimality of NF4        | Advantage of AF4–B         |
|-------------|-------------------------|----------------------------|
| $B\leq64$   | Matches AF4–B           | None                       |
| $B\gg64$    | Suboptimal              | AF4–B yields lower error   |

## 3. Quantization Error Behavior and Empirical Observations

NF4 does not satisfy Lloyd–Max centroid conditions for MSE-minimal quantization. Table 2 of [2505.06653] demonstrates that, for $B=64$ and weight samples from $\mathcal{N}(0,1)$, NF4's MAE and MSE exceed those of both $L_1$-optimized AF4 and MSE-optimized "BOF4" blocks, with BOF4 achieving $\approx$12% lower MSE.

Empirical language modeling results (on LLaMA-7B/13B and others) with $B=64$ indicate negligible differences in perplexity between NF4 and the optimized alternatives. However, as block size increases, the advantage in quantization error of block-optimal designs (e.g., BOF4, AF4–B) translates into slightly better perplexity [2306.06965, 2505.06653].

| Scheme     | MAE $(B=64)$         | MSE $(B=64)$         | WikiText-2 Perplexity Degradation |
|------------|----------------------|----------------------|-----------------------------------|
| NF4        | $9.77\times 10^{-4}$ | $1.64\times 10^{-6}$ | Baseline                         |
| BOF4-S     | $0.954\times 10^{-3}$| $1.44\times 10^{-6}$ | $-$0.07                          |

A plausible implication is that for ultra-large blocks (thousands of weights per block), switching to a block-size-adaptive codebook is desirable for minimizing quantization artifacts.

## 4. Comparisons to Alternative 4-bit Schemes

Compared to standard Int4 and low-bit IEEE-like FP4 variants, NF4 offers:

- Superior near-zero quantization resolution due to nonuniform bin placements.
- Exact representation of block maxima/minima and zero.
- Narrower representable range—tails are more aggressively clipped than in int-based or float-based schemes.

Empirical results demonstrate that NF4 achieves lower perplexity than Int4 and FP4 on benchmark datasets, both in static quantization and after adapter-based finetuning (GLUE, MMLU, etc.), recovering full FP16 performance [2305.14314]. This underpins its widespread adoption in QLoRA and related LLM pipelines.

| Format      | Value Range (per block)  | Zero-shot PPL (Pile-CC)    | Key Drawback               |
|-------------|--------------------------|----------------------------|----------------------------|
| Int4        | $[-c,+c]$ uniform        | 34.34                      | Coarse near zero           |
| FP4 (E2M1)  | $\approx\pm 2c$          | 31.07                      | Coarse quantization        |
| FP4 (E3M0)  | $\approx\pm 8c$          | 29.48                      | No fractional bits         |
| **NF4**     | $[-c,+c]$ coded          | **27.41**                  | Aggressive tail clipping   |

## 5. Double Quantization, Outlier Handling, and Implementation

NF4 in QLoRA further minimizes memory via "double quantization": block scales $c$ are quantized using FP8 floats in secondary blocks (of e.g., 256 scales/block), reducing scale storage to $\approx 0.127$ bits/weight at $B=64$ [2305.14314]. All main MAC operations are performed in higher precision (BF16/FP16) after dequantization.

BOF4-S and OPQ [2505.06653] build directly on NF4 principles. BOF4 utilizes Lloyd/EM algorithms for codebook optimization to minimize end-to-end error, and the signed-absmax normalization (BOF4-S) further reduces error by fixing only the positive endpoint. OPQ (outlier-preserving quantization) stores rare outliers unquantized (e.g., $<$1% in BF16), mitigating the bias induced by extreme values on the scale. This combination further lowers perplexity degradation with negligible memory cost.

## 6. Deployment Guidelines and Practical Recommendations

NF4 is recommended for block sizes up to $B=64$ (default in QLoRA and related toolchains), balancing implementation simplicity, hardware compatibility, and negligible loss relative to theoretically optimal alternatives. For quantizing very large blocks, block-tailored codebooks (AF4–B, BOF4) are preferred [2306.06965, 2505.06653].

Best practices for high-fidelity quantization in LLMs:

- Use block size $B=64$ for weights; store scales in FP8 with double quantization.
- If aiming for minimal quantization error, apply BOF4-S (MSE-optimized) and enable OPQ at $q\approx0.95$.
- Avoid equalizing codebook bin counts, which empirically degrades performance [2306.06965].
- Use standard NF4 for compatibility and performance in standard QLoRA pipelines unless specifically targeting ultra-large blocks.

## 7. Limitations, Ongoing Work, and Theoretical Considerations

NF4 is not information-theoretically optimal for blockwise quantization due to the non-i.i.d. nature of normalized weights post block scaling. Scalar quantization theory (Lloyd–Max) prescribes codebooks adapted to the empirical distribution of normalized weights, motivating recent research into data-driven and block-size-adaptive codebooks. Outlier-aware and mixed-precision quantization schemes (such as OPQ) exploit neural weight distributions to robustly quantize even when extreme values are present.

These developments reflect a trend toward increasingly sophisticated 4-bit quantization strategies, optimizing for a variety of criteria (MSE, MAE, outlier preservation) and deployment constraints (memory, compute, calibration-free operation) in the context of large-scale neural networks [2305.14314, 2306.06965, 2505.06653].

---

**Principal references:**  
- "QLoRA: Efficient Finetuning of Quantized LLMs" [2305.14314]  
- "NF4 Isn't Information Theoretically Optimal (and that's Good)" [2306.06965]  
- "Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations" [2505.06653]

Source: https://www.emergentmind.com/topics/4-bit-normalfloat-nf4