8-bit Quantization Method

• 8-bit quantization is a precision reduction technique that encodes neural network weights, activations, and gradients into 8 bits, enhancing efficiency and conserving memory.
• It employs methods such as uniform-affine integer and low-bit floating-point quantization with per-layer adaptations and stochastic rounding to mitigate quantization noise.
• Empirical results demonstrate minimal accuracy loss with significant speedup and memory savings in models across various tasks and hardware architectures.

8-bit quantization is a precision reduction technique that encodes neural network weights, activations, gradients, and related data structures using 8 bits per element. This approach is widely used for both inference and training of deep neural networks to improve computational throughput, memory footprint, and energy efficiency across diverse hardware platforms, including CPUs, GPUs, NPUs, and embedded devices. Modern methods encompass 8-bit integer and 8-bit floating-point quantization, with algorithmic, statistical, and hardware-driven adaptations that maintain model accuracy even in large-scale, high-performance deployments.

1. Quantization Functions and Core Algorithms

Neural networks quantized to 8 bits leverage a variety of quantization mappings. The two dominant paradigms are uniform-affine integer quantization and low-bit floating-point quantization.

Uniform-Affine Quantization (Integer)

• The quantization function for a real value $x$ in a clamping range $[v_\text{min}, v_\text{max}]$ with $M$ bits is:

$$\text{scale} = \frac{v_\text{max} - v_\text{min}}{2^M - 1}$$

$$\text{zero\_point} = \text{clip}\bigl(\text{round}(-v_\text{min}/\text{scale}), 0, 2^M-1\bigr)$$

$$Q(x) = \text{clip}\bigl(\text{round}(x/\text{scale}) + \text{zero\_point}, 0, 2^M-1\bigr) \times \text{scale}$$

Stochastic rounding is frequently employed to avoid accumulation of quantization bias, especially for gradients and small parameter updates (Banner et al., 2018).

Floating-Point Quantization (FP8)

• FP8 formats such as 1 sign bit, 5 exponent bits, 2 mantissa bits are used, with per-layer or per-tensor exponent bias search to minimize MSE to the source tensor:

$$f = (-1)^s\,2^{p-b}\left(1 + \sum_{i=1}^m \frac{d_i}{2^i}\right)$$

Quantization utilizes clamping and adjustable rounding. Rounding learning may be applied, especially for ultra-low-precision variants (e.g., FP4 weights) (Chen et al., 2024, Mellempudi et al., 2019).

Per-Layer/Group Strategies

• Quantization parameters (scales, clipping) are often computed per-layer, per-channel, or per-group, to better match the dynamic range of each layer or feature (Jin et al., 2022, Dettmers et al., 2022, Guo et al., 2024). Outlier channels may be handled with mixed-precision or decompensation (e.g., LLM.int8()) (Dettmers et al., 2022).

2. Full 8-Bit Training and Inference Frameworks

End-to-end 8-bit quantization pipelines replace all (or nearly all) float32 operations with 8-bit equivalents, including for weights (W), activations (A), gradients (G), errors (E), optimizer updates (U), and normalization statistics (Yang et al., 2019, Banner et al., 2018, Guo et al., 2024).

WAGEUBN

• Provides a unified framework: all major data paths (weights, activations, gradients, errors, updates, batch norm) are quantized to 8 bits where possible. Momentum/optimizer accumulators are quantized, with rare exceptions for error post-BN or accumulation, which sometimes require higher precision (Yang et al., 2019).

INT8 Inference for Transformers

• Integer-only inference is made practical by architectural modifications that replace floating-point softmax with polynomial attention and sqrt-variance normalization with L1-norm normalization, eliminating nearly all float32 ops from the forward path (Lin et al., 2020). Scale propagation manages associated per-tensor scales through computational graphs.

3. Statistical Analysis and Optimization of 8-Bit Formats

Selecting optimal quantization parameters is critical for preserving accuracy:

• Per-Layer Format Selection: Empirical and theoretical analysis shows the optimal fixed-point fractional length (FL) decreases with the dynamic range (standard deviation) of layer activations or weights. F8Net computes per-layer FL via $\mathrm{FL}^* = \lfloor \log_2(c/\sigma) \rfloor$ for constants $c$ specific to signed/unsigned tensors (Jin et al., 2022).
• Quantization Error Bounds: In high dimensions ($N \gtrsim 10^3$), using $M \geq 8$ bits yields cosine similarity between quantized and original weight vectors above $0.99$, implying angular distortion less than $8^\circ$ (Banner et al., 2018).
• Dynamic / Blockwise Quantization: For states with high value variability (e.g., optimizer accumulators), per-block dynamic range normalization and non-linear (tree-based) quantization achieves low average error and robust compression (Dettmers et al., 2021, Dettmers, 2015).

4. Batch Normalization and Normalization Layer Quantization

Range Batch Normalization (Range BN)

• Traditional BatchNorm computes variance, which is sensitive to quantization noise. Range BN instead uses the range statistic $R=\max_i x_i - \min_i x_i$, with an analytically derived scaling factor $\alpha = 2\Phi^{-1}(1-1/N)$ to match standard deviation in the Gaussian limit. Range BN reduces required precision and complexity, empirically yielding $\sim$20% lower latency and 2× greater numerical stability in 8-bit (Banner et al., 2018).

L1-Norm Normalization (L1BNQ/L1LNQ)

• L1-based normalization mitigates sharp loss landscapes and stability issues encountered with L2-norm in quantized networks. The L1 version is provably smoother, yielding lower local Lipschitz constants and robust convergence in low-bit settings. All parameters of the normalization layer are integer-quantized (Guo et al., 2024, Lin et al., 2020).

5. Integer and Floating-Point 8-Bit Variants

Integer-Only Networks

• Fixed-point quantization using $Q_{m.n}$ format enables all-GEMM and convolution operations to use INT8 arithmetic, eliminating the need for dequantization or floating-point accumulation. F8Net demonstrates all-multiplication and accumulation in INT8 via per-layer FL selection (Jin et al., 2022).

FP8/FP8 Quantization

• Many modern accelerators provide identical throughput for INT8 and FP8. FP8 quantization supports substantially wider dynamic range for a given bitwidth and, in context such as diffusion models or transformers, can surpass INT8 in perceptual/noise-tolerance metrics (e.g., FID) without computing cost penalty (Chen et al., 2024).

Optimizer State Quantization

• 8-bit blockwise or dynamic quantization of optimizer statistics (Adam, Momentum) cuts optimizer memory by up to 75% while matching the performance of 32-bit optimizers. Per-block scaling and non-linear mapping are essential for managing the enormous dynamic range in these states (Dettmers et al., 2021).

Method	Weights / Activations	Gradients	BN/Norm	Optimizer	Accuracy Δ (ResNet50/ImageNet)	Memory
Uniform INT8	8/8	8	FP32	FP32	<1% loss (sometimes 3–4%)	4× shrink
WAGEUBN (Yang et al., 2019)	8/8	8	8	8	–5% to –1.5% (see Table)	4× shrink
FP8 (Chen et al., 2024)	8/8	8	8	FP16/32	= or ↑ (vision tasks)	4× shrink
F8Net (Jin et al., 2022)	Q8.FLx, auto				= or ↑ (≤0.3%)	4× shrink
LLM.int8() (Dettmers et al., 2022)	8/8		FP16		0 (massive LLMs)	2× shrink
Blockwise Opt (Dettmers et al., 2021)				8	= or ↑ (NLU, LM)	4–10× opt.

6. Practical Guidelines and Empirical Results

Empirical evidence across vision, language modeling, translation, and speech tasks demonstrates that well-engineered 8-bit quantization achieves near-parity with full-precision baselines:

• 8-bit training and inference of ResNet-50 on ImageNet-1K with uniform-affine quantization and Range BN shows a $<0.2\%$ degradation in Top-1 accuracy ($23.9\%$ 32-bit → $24.1\%$ 8-bit) (Banner et al., 2018).
• All-8-bit integer pipelines (WAGEUBN) yield $69.07\%$ Top-1 vs. $74.66\%$ FP32 (loss $5.6\%$) for ResNet-50; variance primarily attributable to error quantization in later layers (Yang et al., 2019).
• FP8/FP8 quantized diffusion models on CIFAR-10 and LSUN report no significant drop in FID or precision/recall; in some settings, quantization slightly improves image quality metrics (Chen et al., 2024).
• 8-bit optimizers provide 3–9 GB RAM savings on 1B+ parameter models, and a 20–30% speedup in optimizer state updates without any adjustments to base learning rates or schedules (Dettmers et al., 2021).
• LLM.int8() enables inference on 175B parameter models with zero degradation in language modeling perplexity or zero-shot task accuracy, and 1.8× speedup in matmul throughput (Dettmers et al., 2022).

7. Limitations, Best Practices, and Extensions

Best practices distilled from the literature include:

• Always use stochastic rounding for gradients and parameters, especially during training, to avoid bias accumulation and to regularize against quantization noise (Banner et al., 2018, Mellempudi et al., 2019).
• Retain higher precision (e.g., 16- or 32-bit) in first/last layers and master weights, as these layers are most susceptible to quantization-induced performance drops (Banner et al., 2018, Mellempudi et al., 2019, Jin et al., 2022).
• For distributed or model-parallel training, leverage 8-bit compressed communication for gradients and activations to at least double bandwidth efficiency (Dettmers, 2015).
• In normalization, prefer Range BN or L1-based normalization, as they are robust to quantization noise and suppress loss-surface sharpness associated with standard variance (Banner et al., 2018, Guo et al., 2024).
• In deep LLMs, use per-vector or groupwise scales and a mechanism for isolating outlier channels via mixed-precision fallback (Dettmers et al., 2022, Guo et al., 2024).
• For post-training quantization, combine histogram-based or KL-divergence calibrations to set robust thresholds, especially on long-tailed or sparse activations (Bhandare et al., 2019).

Limitations mostly concern catastrophic performance drops with naive quantization in key network layers (softmax, layer norm, embedding), or when hardware support for very low-precision MACs is lacking. Further, sub-8-bit methods often require careful allocation of bits per operator/layer, or compensation techniques to prevent severe accuracy loss (Guo et al., 2024, Miccini et al., 2024).

Contemporary 8-bit quantization, through rigorous design of quantization functions, stochastic error control, statistical adaptation, and hardware-mapped arithmetic, enables robust training and inference with substantial efficiency benefits and near-baseline accuracy across a wide spectrum of neural network architectures (Banner et al., 2018, Yang et al., 2019, Jin et al., 2022, Chen et al., 2024, Guo et al., 2024).