Model Compression & Quantization

• Model compression and quantization are techniques that reduce deep neural network sizes by mapping continuous weights to discrete levels, enabling energy-efficient inference.
• Recent advancements integrate pruning, mixed-precision optimization, and quantization-aware regularization to minimize accuracy loss while dramatically reducing computation and memory requirements.
• Practical guidelines emphasize co-designing quantization with hardware constraints, using sensitivity-based bit allocation and dynamic regularizers to achieve superior trade-offs in modern architectures.

Model compression and quantization are central to enabling efficient inference and deployment of deep neural networks (DNNs) in resource-constrained environments. Compression targets reductions in model size, computation, and energy by exploiting redundancies—most notably, by pruning weights or restricting their numeric representation. Quantization, the mapping of continuous parameters to a finite set of discrete values, underlies nearly all modern compression pipelines and determines the effective bitwidth for both on-device storage and arithmetic. Recent advances integrate quantization with other techniques such as pruning and knowledge distillation, leverage new regularization frameworks, and take into account algorithmic and hardware-level constraints to achieve state-of-the-art trade-offs in accuracy, compression, and efficiency.

1. Fundamentals of Model Quantization and Compression

Quantization methods reduce the precision of neural network weights, activations, or both, mapping real-valued parameters $w$ to a discrete codebook $\mathcal{C}$ with cardinality $K=2^b$ for $b$-bit quantization. Approaches span:

• Uniform quantization: Linear binning in $[–\alpha, +\alpha]$, with step size $\Delta=\alpha/(2^{b-1}-1)$ and projection $\hat w = \textrm{round}(w/\Delta)\cdot\Delta$ (Li et al., 2022).
• Nonuniform and power-of-two quantization: Logarithmically spaced levels capture heavy-tailed or peaky weight distributions, assigning finer granularity near zero (Li et al., 2022, Makenali et al., 4 Sep 2025).
• Adaptive codebooks: Entries learned during training via $k$-means or other clustering, or as direct parameters in quantization-aware regularization (Carreira-Perpiñán et al., 2017, Malchiodi et al., 3 Feb 2026).
• Mixed-precision quantization: Per-parameter, per-layer, or per-channel bitwidths are optimized for hardware or sensitivity (Défossez et al., 2021, Zandonati et al., 2023).

Quantization is often combined with pruning, which removes parameters (weights, filters, or channels) according to magnitude, similarity, or data-driven criteria (Makenali et al., 4 Sep 2025, Tang et al., 2020, Zandonati et al., 2023). Pruning is mathematically equivalent to “quantization to zero” and may be unified with bitwidth allocation in a joint framework (Tang et al., 2020).

Compression ratio is $\approx 32/b$ for weights quantized to $b$ bits, but practical gains depend on redundancy, hardware alignment, and activation quantization. Quantization can be applied post-training (PTQ), during quantization-aware training (QAT), or through differentiable or regularized training to anticipate the discretization step (Défossez et al., 2021, Kundu et al., 2023, Malchiodi et al., 3 Feb 2026).

2. Algorithmic Paradigms and Training Objectives

State-of-the-art quantization schemes modify the learning objective or training pipeline to minimize loss under quantization constraints.

• Augmented Loss Objectives: Add explicit coupling or regularization to induce cluster formation in the weight space during training. For instance, Soft Quantization (Bernstein et al., 29 Jan 2026) augments the base loss with a short-range attractive coupling:

$\mathcal{C}$0

where $\mathcal{C}$1 is a triangular-well potential. This results in emergent clustered weights, yielding discretization without post-hoc quantization.

• Quantization-Aware Regularization: Penalties (e.g., minimum squared distance to trainable cluster centers $\mathcal{C}$2) are added to the loss, driving weights into quantization-friendly configurations during optimization (Malchiodi et al., 3 Feb 2026). Dynamic regularizers (e.g., $\mathcal{C}$3) learn centroids jointly with weights.
• Pseudo-Quantization Noise and Differentiability: Continuous proxies to quantization, such as DiffQ’s uniform noise injection

$\mathcal{C}$4

maintain differentiability w.r.t. both weights and bitwidths, enabling end-to-end optimization (Défossez et al., 2021).

• Vector Loss and Geometry-Aware Optimization: Instead of scalar L2 loss, VecQ introduces a “vector loss” decomposed into angular (orientation) and modulus (scale) components, allowing separate convex optimization over quantization direction and scaling (Gong et al., 2020).
• Pruning-Quantization Integration: Advanced pipelines treat pruning as “0-bit quantization”, enabling joint reinforcement learning or heuristic path-planning based on sensitivity or Fisher Information statistics to allocate both sparsity and bitwidth in a unified search (Zandonati et al., 2023, Tang et al., 2020).

3. Joint Pruning and Quantization Schemes

Best-practices for deep compression now combine structured or unstructured pruning and low-bit quantization. Effective integration achieves multiplicative reductions in both parameter count and per-weight memory:

• Similar Filters Pruning + APoT Quantization: Filters are pruned based on similarity to the geometric median; remaining parameters are quantized via Adaptive Power-of-Two codes for distribution-matching and efficient bit-shifts (Makenali et al., 4 Sep 2025). Both “simultaneous” (SPQ: prune and quantize in each epoch) and “sequential” (PPQ: prune, then quantize) workflows are effective.
• Automated Mixed-Precision Compression: Reinforcement learning or FIM-based heuristics assign per-channel bitwidths and sparsity, optimizing for size, FLOPs, and accuracy under global compression budgets (Tang et al., 2020, Zandonati et al., 2023). The AJPQ algorithm, for instance, uses DDPG to control per-layer sparsity and per-channel bitwidth, mapping pruning and quantization to a shared bitwidth variable.
• Quantization-Aware Pruning for Hardware: Libraries such as PQuantML provide unified pipelines for arbitrarily granular pruning (unstructured, N:M, structured) and per-weight or per-tensor QAT, directly optimizing hardware metrics such as DSP blocks, latency, and EBOP during training (Niemi et al., 27 Mar 2026).

Scheme	Compression Ratio	Accuracy Loss	Method Class
Similarity Pruning + APoT (PPQ)	×15–16	≤1%	Joint (Seq.)
RL Mixed-Precision + Prune (AJPQ)	×5	≤1%–2% (Top-5)	Joint (RL Search)
FIM-based Path Planning (FITCompress)	×30–52	<1% (ImageNet, NL)	Joint (Path Planner)
DST pruning + QAT (PQuantML)	up to ×25	<0.7%	End-to-end HW-aware

4. Regularization for Quantization Robustness

Severe bitwidth reduction exposes quantization error, especially in layers with outliers or poorly suited weight distributions. Several regularization strategies have been proposed:

• Range Restriction Loss (R²-Loss): Penalizes weight outliers during full-precision training, tightening the support and improving post-training or QAT quantization in 1-2 bit regimes (Kundu et al., 2023).
• Weight Normalization-Based Quantization (WNQ): Normalizes each filter by its maximal absolute element before quantization, effectively suppressing long tails and reducing relative quantization error (Cai et al., 2019).
• Hyperspherical Quantization (HQ): Constrains weights to a sphere and performs iterative prune–reinit–quantize steps, directly minimizing cosine distance between ternary and full-precision vectors to reduce STE bias and maintain accuracy even at 2-bit precision (Liu et al., 2022).
• Periodicity-Inducing Regularizers: Sine or cosine-based losses introduce basin structures aligned with intended quantization levels, making the post-quantization mapping less lossy (Malchiodi et al., 3 Feb 2026).

5. Post-Training Quantization, Mixed Precision, and Hardware Targets

For inference-only scenarios or deployment on diverse hardware, post-training quantization (PTQ) and hybrid precision methods are dominant:

• Rotation-Invariant Quantization (RIQ): Proposes a single-parameter, layer-wise mixed-precision quantizer derived from geometric rate–distortion arguments. RIQ theoretically minimizes rate for a given distortion by allocating bin width $\mathcal{C}$5 layer-wise (Kampeas et al., 2023).
• Standardized Toolkit Benchmarks (LLMC): LLMC benchmarks quantization algorithms across LLMs under varying data calibrations, clipping strategies, search-based scaling, and precision allocation (per-tensor, per-column). Empirically, search-based asymmetric clipping and per-channel scaling (TS-v1 + CS-asym) are practical defaults, while Hessian-based mixed-precision guides per-column allocation in transformer blocks (Gong et al., 2024).
• Structured Compression Formats: For aggressive pruning, nonstandard encodings such as weight encryption via XOR-coding lead to regular, parallelizable decoding with $\mathcal{C}$60.28 bits/weight at 91% pruning and 1-bit quantization, outperforming CSR or Viterbi encoding for high-sparsity blocks (Kwon et al., 2019).
• Hardware-Aware Objectives: Integration of EBOP (effective bit-operations), per-layer hardware budgets, or direct latency constraints into training pipelines closes the algorithm–hardware loop (Niemi et al., 27 Mar 2026).

6. Theoretical and Empirical Insights into Compression–Generalization Trade-offs

Emergent phenomena in highly compressed neural networks challenge previous assumptions:

• High-Dimensional Redundancy: Networks remain robust to quantization and pruning far beyond naive expectation by realigning solutions along “flat” directions of the loss landscape, as seen in Soft Quantization (Bernstein et al., 29 Jan 2026).
• Compression Helmholtz: Regularization and joint optimization smooth quantization error evolution, enabling transitions into quantized minima without crossing high-loss barriers (Bernstein et al., 29 Jan 2026). Compression behavior is heavily correlated with the geometric structure of the loss surface.
• Layer and Architecture Sensitivity: Distribution shape, parameter density, and bit-allocation must be tuned to specific architectures (e.g., ResNet vs. ECAPA-TDNN) for robustness (Li et al., 2022).

7. Practical Guidelines, Limitations, and Future Directions

• Mixed-precision, pruning, and quantization should be co-designed, using either sensitivity (FIM/gradient-based) or learning-based allocation, rather than applied sequentially.
• Aggressive range restriction and normalization are essential for ultra-low-bit (≤2-bit) quantization; otherwise, outliers erode effective dynamic range.
• Quantization-aware training or regularization offers substantial gains over post-hoc quantization for modern high-capacity architectures.
• Hardware-aware and constraint-driven pipelines are increasingly critical; integration of latency, DSP, and memory metrics during optimization is available in libraries such as PQuantML.
• Most schemes are compatible with both classification, detection, and sequence models, but layer-specific tuning (including for non-convolutional, e.g., transformer blocks) remains necessary.
• Limitations persist in regimes of extreme quantization (<2 bits), large pruning, and for highly irregular architectures; future work aims to refine loss-aware bit allocation, incorporate hardware-in-the-loop optimization, and extend principles to activations and gradients.

Model compression and quantization continue to evolve from heuristic post-training projections to mathematically grounded, hardware-aware, and end-to-end optimized modules, enabling scalable deployment of increasingly large and capable deep learning models (Bernstein et al., 29 Jan 2026, Makenali et al., 4 Sep 2025, Défossez et al., 2021, Zandonati et al., 2023, Kundu et al., 2023, Liu et al., 2022, Malchiodi et al., 3 Feb 2026, Niemi et al., 27 Mar 2026, Gong et al., 2024).