QuEPT: Error Propagation in PTQ

• The paper introduces a framework for layer-wise PTQ that explicitly models, propagates, and compensates quantization errors across layers.
• It formulates a closed-form corrective target using Hessian inversion and a tunable propagation coefficient to address error accumulation in low-bit quantization.
• Experimental results on LLaMA2 models show significant improvements in perplexity and zero-shot accuracy, especially in extremely low-bit settings.

QuEPT is a discussion name for Quantization Error Propagation (QEP), a framework for layer-wise post-training quantization (PTQ) of LLMs that explicitly models, propagates, and compensates quantization errors accumulated across layers. In the formulation introduced in “Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization,” QuEPT revisits the standard sequential PTQ pipeline, identifies growth of upstream quantization error as a central low-bit failure mode, and replaces purely local layer objectives with an error-aware correction mechanism that remains compatible with standard PTQ solvers such as RTN, GPTQ, AWQ, and rotation-based methods (Arai et al., 13 Apr 2025).

1. Position within layer-wise post-training quantization

Conventional layer-wise PTQ quantizes linear operations sequentially and optimizes each layer independently. For the $l$-th linear operation with weight matrix $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$ and calibration inputs $\mathbf{X}\in\mathbb{R}^{d\times m}$, the discrete quantized weight is $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$, where $\mathbb{Q}\subset\mathbb{R}$ denotes the quantized value set. The standard objective is

$$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$

where $\mathsf{X}_l$ is either the quantized upstream input $\hat{\mathbf{X}}_l$ or the full-precision upstream input $\mathbf{X}_l$. The associated quadratic form yields a Hessian $\mathsf{H}_l=\mathsf{X}_l\mathsf{X}_l^\top$, which many PTQ methods reuse for efficiency (Arai et al., 13 Apr 2025).

The central empirical observation behind QuEPT is that early-layer quantization errors do not remain local. With unquantized Transformer blocks denoted $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$0 and quantized blocks $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$1, the error at block $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$2 is defined as

$\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$3

with

$\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$4

When only the first $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$5 blocks of LLaMA2-7B are quantized and deeper blocks remain full precision, $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$6 continues to grow approximately exponentially with depth. This identifies a core bottleneck of ordinary layer-wise PTQ: local objectives do not account for global error dynamics (Arai et al., 13 Apr 2025).

2. Formalization of propagated quantization error

QuEPT is formulated for weight-only quantization. The framework is orthogonal to the exact quantizer type and was evaluated under per-channel and group-wise settings including INT4, INT3, INT2, and group-wise variants such as INT3g128, INT2g32, INT2g64, and INT2g128 (Arai et al., 13 Apr 2025).

The upstream full-precision and quantized inputs at layer $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$7 are denoted $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$8 and $\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}$9. Their discrepancy is

$\mathbf{X}\in\mathbb{R}^{d\times m}$0

This term captures the cumulative activation perturbation induced by previously quantized layers. In ordinary layer-wise PTQ, the current layer is still optimized against a shared input surrogate, which obscures the distinction between the target full-precision response and the quantized-input response actually presented at inference time. QuEPT replaces that local matching problem with

$\mathbf{X}\in\mathbb{R}^{d\times m}$1

The reformulation is significant because it makes the compensation target depend explicitly on accumulated upstream error. Rather than matching full-precision and quantized weights under the same calibration input, the method asks the quantized layer to approximate the full-precision output under the input it will actually receive after preceding layers have been quantized (Arai et al., 13 Apr 2025).

A plausible implication is that QuEPT should become progressively more valuable as quantization bit-width decreases, because the discrepancy $\mathbf{X}\in\mathbb{R}^{d\times m}$2 becomes larger and more structurally important in low-bit regimes. The reported experiments are consistent with that interpretation (Arai et al., 13 Apr 2025).

3. Corrective target, closed-form compensation, and tunable propagation

The key theoretical step is to relax the discrete feasible set from $\mathbf{X}\in\mathbb{R}^{d\times m}$3 to $\mathbf{X}\in\mathbb{R}^{d\times m}$4. Defining the empirical Hessian of quantized inputs as

$\mathbf{X}\in\mathbb{R}^{d\times m}$5

the relaxed problem admits a closed-form optimal corrective weight,

$\mathbf{X}\in\mathbb{R}^{d\times m}$6

This formula shows that the correction consists of a propagated compensation term determined jointly by upstream error $\mathbf{X}\in\mathbb{R}^{d\times m}$7 and local curvature through $\mathbf{X}\in\mathbb{R}^{d\times m}$8 (Arai et al., 13 Apr 2025).

Returning to discrete quantization, QuEPT re-casts the layer subproblem into the same quadratic structure used by standard PTQ methods, but with the corrected target $\mathbf{X}\in\mathbb{R}^{d\times m}$9:

$\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$0

This plug-and-play structure is a defining feature. It means that existing PTQ optimizers do not need to be replaced; they are retargeted from $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$1 to $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$2 (Arai et al., 13 Apr 2025).

To regulate overfitting and computational cost, QuEPT introduces a per-layer propagation coefficient $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$3:

$\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$4

Two limiting cases are explicit. When $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$5, the layer applies full correction. When $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$6, the method reduces to the conventional objective with $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$7, i.e., no propagation. The paper recommends smaller $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$8, or even $\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}$9, for parameter-heavy components such as MLP blocks in order to reduce computation and regularize against overfitting on small calibration sets. For numerical stability, QuEPT damps the Hessian by adding a diagonal term $\mathbb{Q}\subset\mathbb{R}$0 equal to the mean of $\mathbb{Q}\subset\mathbb{R}$1 before inversion, following GPTQ practice (Arai et al., 13 Apr 2025).

4. Integration into the PTQ pipeline

QuEPT is applied at the level of individual linear operations, not whole Transformer blocks. It therefore covers attention and MLP linear layers within each block and extends naturally to any sequential layer-wise PTQ pipeline (Arai et al., 13 Apr 2025).

The procedure is organized in two stages. First, a full-precision calibration pass collects $\mathbb{Q}\subset\mathbb{R}$2 for each layer by forwarding calibration data through layers $\mathbb{Q}\subset\mathbb{R}$3 in full precision. Second, quantization proceeds sequentially from input to output. At each layer $\mathbb{Q}\subset\mathbb{R}$4, the method computes quantized upstream inputs $\mathbb{Q}\subset\mathbb{R}$5, forms the upstream discrepancy $\mathbb{Q}\subset\mathbb{R}$6, computes $\mathbb{Q}\subset\mathbb{R}$7, applies damping if necessary, constructs $\mathbb{Q}\subset\mathbb{R}$8, and solves

$\mathbb{Q}\subset\mathbb{R}$9

using a standard PTQ solver (Arai et al., 13 Apr 2025).

The solver-specific interpretation is deliberately minimal. In RTN, one applies round-to-nearest directly to $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$0. In GPTQ, one reuses $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$1 and its inverse, performs row-wise quantization with residual correction, but targets $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$2. In AWQ, one computes saliency or rescaling as usual, but on $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$3 rather than $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$4. Rotation-based methods such as QuIP, QuaRot, and SpinQuant are treated as orthogonal: QuEPT corrects propagated upstream errors, whereas those methods homogenize weight magnitudes through orthogonal transformations (Arai et al., 13 Apr 2025).

The additional computation beyond ordinary PTQ is dominated by forming the correction term $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$5. Because $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$6 is typically already available in Hessian-based PTQ pipelines, the principal new cost is $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$7, which the paper characterizes as negligible at scale in practice. Setting $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$8 on selected layers removes this overhead there entirely (Arai et al., 13 Apr 2025).

5. Experimental behavior and performance profile

The reported evaluation covers LLaMA2-7B, 13B, and 70B. Calibration uses 128 random 2048-token segments from C4. Main metrics are WikiText2 perplexity, with PTB and C4 in the appendix, and zero-shot accuracy averaged over ARC Easy, PiQA, and StoryCloze (Arai et al., 13 Apr 2025).

The strongest pattern is that QuEPT is most effective in very low-bit settings. Representative WikiText2 perplexity results are summarized below.

Setting	Baseline $$\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}} \left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,$$9 QuEPT	Metric
LLaMA2-7B, RTN, INT2g32	90.692 $\mathsf{X}_l$0 12.249	PPL
LLaMA2-7B, RTN, INT2g128	4270.828 $\mathsf{X}_l$1 35.291	PPL
LLaMA2-70B, RTN, INT2g128	27.268 $\mathsf{X}_l$2 8.799	PPL
LLaMA2-7B, GPTQ, INT3	10.881 $\mathsf{X}_l$3 7.898	PPL
LLaMA2-70B, AWQ, INT3g128	3.740 $\mathsf{X}_l$4 3.724	PPL

The paper also reports INT4 refinements, including LLaMA2-7B GPTQ INT4 from 6.083 to 5.933 and AWQ INT4 from 5.831 to 5.756. Group-wise INT3g128 and INT4g128 remain strong baselines, with QuEPT still improving or matching the best reported values. This suggests that the framework is not limited to rescuing weak baselines; it can refine already competitive PTQ settings (Arai et al., 13 Apr 2025).

Zero-shot averages improve in the same general direction. For INT2g128 with RTN, LLaMA2-7B rises from 0.4296 to 0.5598, 13B from 0.4834 to 0.5875, and 70B from 0.5593 to 0.6470. For GPTQ INT3, the corresponding averages increase from 0.6367 to 0.6549 on 7B, from 0.6747 to 0.6853 on 13B, and from 0.7043 to 0.7078 on 70B (Arai et al., 13 Apr 2025).

Not every reported cell is monotone. One GPTQ INT2 entry for LLaMA2-13B increases WikiText2 perplexity from 1301.395 to (2782.3528), whereas the 7B and 70B GPTQ INT2 entries improve substantially. This suggests that the empirical advantage is strong overall but not uniform across every model-solver-bit combination (Arai et al., 13 Apr 2025).

Runtime measurements on a single NVIDIA V100 GPU indicate modest or negligible overhead. Full-model quantization times are reported as 14.9 minutes, 26.4 minutes, and 2.9 hours for GPTQ on 7B, 13B, and 70B; 13.6 minutes, 25.4 minutes, and 2.4 hours for AWQ; and 10.9 minutes, 19.6 minutes, and 1.7 hours for QEP + RTN. The paper attributes this to reuse of calibration computations (Arai et al., 13 Apr 2025).

Robustness experiments on LLaMA2-7B INT3g128 further indicate reduced sensitivity to calibration data. Relative to RTN, GPTQ changes WikiText2 perplexity by $\mathsf{X}_l$5 on C4, $\mathsf{X}_l$6 on PTB, and $\mathsf{X}_l$7 on WT2, whereas QEP+RTN changes it by $\mathsf{X}_l$8, $\mathsf{X}_l$9, and $\hat{\mathbf{X}}_l$0, respectively. The PTB degradation observed under GPTQ does not appear in the QuEPT-enhanced variant (Arai et al., 13 Apr 2025).

6. Practical usage, scope, limitations, and nomenclature

The practical guidance in the paper is explicit. For low-bit regimes, especially 2–3 bits, group-wise quantization such as INT2g32, INT2g64, or INT2g128 is preferred, and QuEPT is presented as particularly impactful there. The recommended default is $\hat{\mathbf{X}}_l$1 for attention layers and early blocks, with $\hat{\mathbf{X}}_l$2 or $\hat{\mathbf{X}}_l$3 for selected MLP blocks to reduce compute and mitigate overfitting. The damping parameter is set to $\hat{\mathbf{X}}_l$4. The same calibration data should be used for QuEPT and baseline PTQ in order to minimize preprocessing differences (Arai et al., 13 Apr 2025).

Several limitations are stated directly. QuEPT is weight-only and layer-wise; it does not address activation quantization or joint multi-layer optimization. Calibration data are assumed to be available, and their quality and domain match still matter even though robustness is improved. Hessian inversion may remain costly for very large input dimensions, although damping and selective $\hat{\mathbf{X}}_l$5 mitigate part of that cost. Adaptive, data-aware strategies for choosing $\hat{\mathbf{X}}_l$6 per layer or per block remain open (Arai et al., 13 Apr 2025).

The acronym itself is not stable across the literature. In the context of (Arai et al., 13 Apr 2025), “QuEPT” refers informally to QEP, or Quantization Error Propagation. Elsewhere on arXiv, however, QuEPT can denote the unrelated “Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching” (Xu et al., 13 Feb 2026), while QuEPP denotes “Quantum Enhanced Pauli Propagation” (Majumder et al., 15 Mar 2026). This naming overlap is a source of potential confusion, but the method discussed here is the layer-wise PTQ framework centered on propagated quantization-error compensation (Arai et al., 13 Apr 2025).

In its intended sense, QuEPT is best understood as a reparameterization of the layer-wise PTQ target rather than a new standalone quantizer. Its contribution is to expose accumulated upstream discrepancy as an explicit optimization variable, convert that discrepancy into a closed-form corrective weight, and retain the quadratic solver structure required by widely used PTQ methods. That combination explains why its gains are largest where local layer objectives are least adequate: the extremely low-bit regime (Arai et al., 13 Apr 2025).