Quantized LLMs: Advanced Techniques

Updated 26 March 2026

Quantized LLMs are neural networks that use low-bit precision for weights and activations, significantly reducing computational and memory overhead.
They employ advanced techniques such as post-training quantization, quantization-aware training, and adaptive mixed-precision allocation to optimize performance.
Empirical studies show minimal accuracy loss at 4-bit quantization, enabling practical deployment on edge and resource-constrained platforms.

Quantized LLMs are a class of neural network models in which the weights, and often activations and key-value caches, are represented with reduced numerical precision—specifically, with low bit-width integer or floating-point formats (typically 8, 4, 3, 2, or even 1 bit per value) rather than standard FP32 or FP16. This quantization process yields substantial reductions in model memory and computational cost while enabling deployment on edge devices, consumer GPUs, and serving accelerators, often with minimal loss in language modeling accuracy or task performance (Lang et al., 2024, Gong et al., 2024). The design, calibration, and downstream behavior of quantized LLMs constitute a rapidly maturing research area, with methods spanning post-training quantization (PTQ), quantization-aware training (QAT), advanced codebooks, mixed-precision allocation, and loss-aware post-processing.

1. Mathematical Principles of Quantization in LLMs

Given a full-precision neural weight $w \in \mathbb{R}$ , quantization maps $w$ to a discrete set of $K=2^b$ levels via a quantizer function $Q$ . The canonical affine quantizer (uniform symmetric or asymmetric) is:

$w_q = \mathrm{clip}(\mathrm{round}((w - z) / s), q_\mathrm{min}, q_\mathrm{max}), \quad \hat{w} = s \, w_q + z$

where $s$ is the scale (step size), $z$ is the zero-point (for asymmetric quantization), and $b$ is the target bit-width. The calibration of $s$ and $z$ typically minimizes maximal ( $L_\infty$ ) or mean-squared ( $L_2^2$ ) quantization error, with either statistical or task-driven objectives (Lang et al., 2024).

For LLMs, quantization may be applied per-layer, per-group, or per-channel, and both weights and activations can be quantized. More advanced schemes include non-uniform codebooks via $k$ -means (Zhang et al., 2024), additive multi-codebook compositions (Giagnorio et al., 10 Mar 2025), or convex-programmed global bit allocation (Young, 2024).

2. Algorithms: Post-Training and Quantization-Aware Methods

2.1. Post-Training Quantization (PTQ)

PTQ methods freeze model weights and apply quantization as a separate step. Notable PTQ algorithms include:

RTN (Round-to-Nearest): Simple uniform/affine mapping, typically per-channel, using scales derived from min/max of calibration data or statistical percentiles (Lang et al., 2024, Roy, 2023).
GPTQ: Block-wise second-order error minimization using block-wise (Cholesky-inverted) Hessian approximations to locally minimize mean-square output error via iteratively updated quantizations (Lang et al., 2024, Zhang et al., 2024, Xu et al., 2024).
AWQ/SmoothQuant: Rebalances activation and weight scales to mitigate the impact of activation outliers, often using affine transformations prior to quantization (Lang et al., 2024, Liu et al., 2023).
Layer/group-wise mixed-precision: Automatically allocates higher bit-width to sensitive or important layers or channels, sometimes via Hessian traces or semantic importance metrics (Zeng et al., 2024, Young, 2024).

2.2. Advanced and Extreme PTQ

Advanced approaches address the limitations when aggressively pushing bit-widths below 4:

Adaptive Channel Reassembly: In QLLM, activation outlier channels are split and merged to improve quantization ranges, followed by lightweight low-rank error correction (Liu et al., 2023).
Additive Quantization (AQLM): Each group of weights is reconstructed as a sum from multiple learned low-bit codebooks, enabling high compression at 2–4 bits with minimal loss (Giagnorio et al., 10 Mar 2025, Fu et al., 26 Aug 2025).
Sigma-Delta and Hadamard-based Smoothing: SDQ-LLM leverages oversampled sigma-delta binarization, combined with Hadamard rotations and continuous OSR adjustment, to enable 1–1.58 bit quantization (Xia et al., 27 Sep 2025).
Binary Quantization with Dynamic Grouping: Irregular, variational grouping and optimal group-wise bit allocation achieves near-four-bit performance at ∼1 bit per weight (Zheng et al., 3 Sep 2025).

2.3. Quantization-Aware Training (QAT)

QAT introduces fake quantization nodes in the forward/backward passes during fine-tuning or instruction alignment, using straight-through estimators (STE) to propagate gradients through discrete quantization steps (Lang et al., 2024). Modern QAT often incorporates mixed-precision LoRA adapters or alignment objectives (e.g., DPO/QLORA) to maintain performance at ultra-low bit-widths (Yi et al., 2024, Lee et al., 2024).

3. Calibration Data and Optimization Strategies

The quantization performance of LLMs is highly sensitive to the calibration protocol:

Calibration Dataset: A small, task-representative or domain-specific dataset (e.g., 128–2048 sequences of length 512–2048) is recommended. For domain-specialized LLMs (e.g., code), code-focused calibration is essential below 4 bits (Gong et al., 2024, Giagnorio et al., 10 Mar 2025).
Loss/Gradient-Aware Calibration: LeanQuant and GWQ use inverse-Hessian diagonals or first-order gradients to construct loss-aware (nonuniform) grids or identify outlier weights for preferential retention in higher precision (Shao et al., 2024, Zhang et al., 2024).
Adaptive Bit Allocation: Layer-specific schemes such as LSAQ use semantic metrics (e.g., Jaccard similarity between top- $k$ tokens pre/post-layer) to dynamically downshift precision in low-importance layers, maximizing compression under resource constraints (Zeng et al., 2024).

4. Practical Effects: Accuracy, Efficiency, and Trade-offs

4.1. Memory and Latency Gains

INT8 or 8-bit quantization typically halves model memory and supports linear algebra acceleration, with negligible accuracy drop (Lang et al., 2024, Roy, 2023).
INT4/FP4/NF4 formats reduce memory by 60–70%, with only minor losses in perplexity and end-task accuracy for models up to 70B parameters. Double quantization (dq) further compresses by 5–10% but with increased compute overhead (Roy, 2023).
Mixed-precision and advanced codebook methods push average bit-width to 2–3 with rapid memory cost decay; extremely aggressive 1–2 bit quantization is possible at the expense of notable accuracy degradation unless using advanced methods (Giagnorio et al., 10 Mar 2025, Xia et al., 27 Sep 2025, Zheng et al., 3 Sep 2025).

4.2. Accuracy, Degradation, and Scaling Laws

For standard PTQ, 4-bit weight-only quantization is a consistent "safe zone": LLMs (Llama, OPT, Falcon, etc.) maintain perplexity within 0.3 of FP16, and code-generation or QA pass@1 is degraded by ≤3% (Lang et al., 2024, Giagnorio et al., 10 Mar 2025).
Performance deteriorates rapidly below 4 bits unless using domain- or loss-aware methods. For 2–3 bit quantization, domain-matched calibration and post-quantization fine-tuning (e.g., AQLM with PV-tuning, QLLM low-rank correction) can recover substantial lost accuracy (Liu et al., 2023, Giagnorio et al., 10 Mar 2025).
Scaling law analysis reveals that the signal-to-quantization-noise ratio (SQNR) is predictable from the bit-width and that a sharp loss jump occurs below ≈20dB SQNR (∼2–3 bits). GPTQ PTQ yields maximal benefit in the 10–20dB SQNR regime; at very high or very low SQNR, further optimization is less effective (Xu et al., 2024).
CVXQ provides an optimal mixed-precision allocation per group/layer under a bit budget via convex programming, achieving state-of-the-art (SOTA) rate–distortion performance at all scale points (Young, 2024).

4.3. Specialized Effects: Truthfulness, Conversational Ability, and Outlier Sensitivity

Quantized models preserve internal truth features but are more susceptible to adversarial prompt steering (e.g., deceptive prompts can induce models to output falsehoods even if internal truth separation is intact) (Fu et al., 26 Aug 2025).
Instruction alignment and conversational abilities of quantized chatbots can substantially degrade, especially under token-flip errors when the output logit margin is small. Post-quantization direct preference optimization (QDPO) effectively realigns quantized models to restore human-judged dialogue quality (Lee et al., 2024).
Outlier handling is a key determinant of low-bit quantization success; channel reassembly, rotation-based smoothing, and advanced grouping address outlier effects in both activations and weights, stabilizing accuracy under extreme compression (Xiao et al., 27 Nov 2025, Liu et al., 2023).

5. Specialized Approaches for Ultra-Low Bit Quantization

SDQ-LLM introduces a hybrid pipeline (upsampling, sigma-delta quantization, Hadamard rotation, and OSR allocation) enabling dynamic trade-off between 1–2 bits per weight and task accuracy. This approach preserves sign-only matrix calculation and achieves ∼20% FP16 storage at <+5 PPL loss for large models.

By relaxating the block-group constraint to allow unstructured groupings (found via dynamic programming or efficient greedy windowed merging), SOTA 1-bit quantization approaches achieve near-4-bit performance (e.g., LLaMA3.2-3B: PPL 8.23 @ 1.007 bits/weight).

LeanQuant replaces non-adaptive min–max grids with error-aware, inverse-Hessian-weighted grids, solving a $k$ -means problem per block. Empirically, LeanQuant outperforms uniform-grid GPTQ and OmniQuant, especially at 2–3 bits.

6. Fine-Tuning, Alignment, and One-Shot/Multi-Deployable Quantized LLMs

QuZO enables forward-only, unbiased zeroth-order (no-backprop) INT8/4 quantized fine-tuning via stochastic rounding and two-quantization random gradient estimation, with convergence and stability guarantees (Zhou et al., 17 Feb 2025).
The "once-for-all" (LLM-QFA) paradigm extends quantization to deployment heterogeneity: LoRA adapters are learned per-bit-width while weights are decoupled, enabling mixed-precision subnet extraction with a single training pass (Yi et al., 2024).
Preference alignment and calibration-aware alignment, such as QDPO, are essential for conversational LLMs post quantization, bridging the gap between statistical accuracy and human-judged quality (Lee et al., 2024).

7. Open Challenges and Research Frontiers

Achieving reliable performance in ultra-low bit regimes (1–2 bit) remains nontrivial; novel codebook designs, multi-granularity adaptive grouping, and error-aware calibration are current focuses (Zhang et al., 2024, Xia et al., 27 Sep 2025, Zheng et al., 3 Sep 2025).
Generalization beyond language (e.g., vision–language or multimodal transformers) and dynamic/online quantization under non-stationary input distributions remain largely open (Shao et al., 2024).
End-to-end quantization pipelines that support on-device alignment, domain-aware adaptation, and live resource tracking (dynamic quantization under memory constraints) are becoming increasingly salient (Zeng et al., 2024, Young, 2024).
Preference-aware and value-aligned quantized models, including those robust to adversarial prompt distribution shifts or internal misalignment, are active areas of exploration (Fu et al., 26 Aug 2025, Lee et al., 2024).

In summary, quantized LLMs now routinely achieve substantial memory and speed gains across a spectrum of hardware and use cases, with 4-bit PTQ representing a reliable baseline for most transformer families. Innovations in gradient/loss-aware quantizers, mixed-precision allocation, and group/coding strategies are steadily advancing the frontier toward efficient, robust, and versatile deployment at the edge, in the data center, and on resource-constrained platforms (Lang et al., 2024, Zhang et al., 2024, Young, 2024, Xiao et al., 27 Nov 2025, Giagnorio et al., 10 Mar 2025).