---
title: 'QuEPT: Error Propagation in PTQ'
url: https://www.emergentmind.com/topics/quept
type: topic
---

# QuEPT: Error Propagation in PTQ

QuEPT is a discussion name for **Quantization Error Propagation (QEP)**, a framework for layer-wise post-training quantization (PTQ) of large language models that explicitly models, propagates, and compensates quantization errors accumulated across layers. In the formulation introduced in “Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization,” QuEPT revisits the standard sequential PTQ pipeline, identifies growth of upstream quantization error as a central low-bit failure mode, and replaces purely local layer objectives with an error-aware correction mechanism that remains compatible with standard PTQ solvers such as RTN, GPTQ, AWQ, and rotation-based methods [2504.09629].

## 1. Position within layer-wise post-training quantization

Conventional layer-wise PTQ quantizes linear operations sequentially and optimizes each layer independently. For the \(l\)-th linear operation with weight matrix \(\mathbf{W}_l \in \mathbb{R}^{n_l \times d_l}\) and calibration inputs \(\mathbf{X}\in\mathbb{R}^{d\times m}\), the discrete quantized weight is \(\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}\), where \(\mathbb{Q}\subset\mathbb{R}\) denotes the quantized value set. The standard objective is

\[
\min_{\hat{\mathbf{W}}_l \in \mathbb{Q}^{n_l\times d_l}}
\left\| \mathbf{W}_l \,\mathsf{X}_l - \hat{\mathbf{W}}_l \,\mathsf{X}_l \right\|_F^2,
\]

where \(\mathsf{X}_l\) is either the quantized upstream input \(\hat{\mathbf{X}}_l\) or the full-precision upstream input \(\mathbf{X}_l\). The associated quadratic form yields a Hessian \(\mathsf{H}_l=\mathsf{X}_l\mathsf{X}_l^\top\), which many PTQ methods reuse for efficiency [2504.09629].

The central empirical observation behind QuEPT is that early-layer quantization errors do not remain local. With unquantized Transformer blocks denoted \(\mathtt{TransBlock}_{m}(\cdot)\) and quantized blocks \(\widehat{\mathtt{TransBlock}}_{m}(\cdot)\), the error at block \(m\) is defined as

\[
\Delta_{m} = \left\| f_m(\mathbf{X}) - \hat{f}_m(\mathbf{X})\right\|_F^2,
\]

with

\[
f_m(\mathbf{X}) \delequal \mathtt{TransBlock}_m \circ \cdots \circ \mathtt{TransBlock}_1(\mathbf{X}), \qquad
\hat{f}_m(\mathbf{X}) \delequal \mathtt{TransBlock}_m \circ \cdots \circ \widehat{\mathtt{TransBlock}}_1(\mathbf{X}).
\]

When only the first \(n=10\) blocks of LLaMA2-7B are quantized and deeper blocks remain full precision, \(\Delta_m\) continues to grow approximately exponentially with depth. This identifies a core bottleneck of ordinary layer-wise PTQ: local objectives do not account for global error dynamics [2504.09629].

## 2. Formalization of propagated quantization error

QuEPT is formulated for **weight-only quantization**. The framework is orthogonal to the exact quantizer type and was evaluated under per-channel and group-wise settings including INT4, INT3, INT2, and group-wise variants such as INT3g128, INT2g32, INT2g64, and INT2g128 [2504.09629].

The upstream full-precision and quantized inputs at layer \(l\) are denoted \(\mathbf{X}_l\) and \(\hat{\mathbf{X}}_l\). Their discrepancy is

\[
\mathbf{\delta}_l \delequal \mathbf{X}_l - \hat{\mathbf{X}}_l.
\]

This term captures the cumulative activation perturbation induced by previously quantized layers. In ordinary layer-wise PTQ, the current layer is still optimized against a shared input surrogate, which obscures the distinction between the target full-precision response and the quantized-input response actually presented at inference time. QuEPT replaces that local matching problem with

\[
\min_{\hat{\mathbf{W}}_l\in\mathbb{Q}^{n_l\times d_l}}
\left\| \mathbf{W}_l\,\mathbf{X}_l - \hat{\mathbf{W}}_l\,\hat{\mathbf{X}}_l \right\|_F^2.
\]

The reformulation is significant because it makes the compensation target depend explicitly on accumulated upstream error. Rather than matching full-precision and quantized weights under the same calibration input, the method asks the quantized layer to approximate the full-precision output under the input it will actually receive after preceding layers have been quantized [2504.09629].

A plausible implication is that QuEPT should become progressively more valuable as quantization bit-width decreases, because the discrepancy \(\mathbf{\delta}_l\) becomes larger and more structurally important in low-bit regimes. The reported experiments are consistent with that interpretation [2504.09629].

## 3. Corrective target, closed-form compensation, and tunable propagation

The key theoretical step is to relax the discrete feasible set from \(\mathbb{Q}^{n_l\times d_l}\) to \(\mathbb{R}^{n_l\times d_l}\). Defining the empirical Hessian of quantized inputs as

\[
\hat{\mathbf{H}}_l \delequal \hat{\mathbf{X}}_l\hat{\mathbf{X}}_l^\top,
\]

the relaxed problem admits a closed-form optimal corrective weight,

\[
\mathbf{W}_l^{\ast} \delequal \mathbf{W}_l + \mathbf{W}_l\,\mathbf{\delta}_l\,\hat{\mathbf{X}}_l^\top\,\hat{\mathbf{H}}_l^{-1}.
\]

This formula shows that the correction consists of a propagated compensation term determined jointly by upstream error \(\mathbf{\delta}_l\) and local curvature through \(\hat{\mathbf{H}}_l^{-1}\) [2504.09629].

Returning to discrete quantization, QuEPT re-casts the layer subproblem into the same quadratic structure used by standard PTQ methods, but with the corrected target \(\mathbf{W}_l^\ast\):

\[
\min_{\hat{\mathbf{W}}_l\in \mathbb{Q}^{n_l\times d_l}}
\left\| \mathbf{W}_l^{\ast}\,\hat{\mathbf{X}}_l - \hat{\mathbf{W}}_l\,\hat{\mathbf{X}}_l \right\|_F^2.
\]

This plug-and-play structure is a defining feature. It means that existing PTQ optimizers do not need to be replaced; they are retargeted from \(\mathbf{W}_l\) to \(\mathbf{W}_l^\ast\) [2504.09629].

To regulate overfitting and computational cost, QuEPT introduces a per-layer propagation coefficient \(\alpha_l \in [0,1]\):

\[
\mathbf{W}_l^{\ast}(\alpha_l) =
\mathbf{W}_l + \alpha_l\,\mathbf{W}_l\,\mathbf{\delta}_l\,\hat{\mathbf{X}}_l^\top\,\hat{\mathbf{H}}_l^{-1}.
\]

Two limiting cases are explicit. When \(\alpha_l=1\), the layer applies full correction. When \(\alpha_l=0\), the method reduces to the conventional objective with \(\mathsf{X}_l=\hat{\mathbf{X}}_l\), i.e., no propagation. The paper recommends smaller \(\alpha_l\), or even \(0\), for parameter-heavy components such as MLP blocks in order to reduce computation and regularize against overfitting on small calibration sets. For numerical stability, QuEPT damps the Hessian by adding a diagonal term \(\lambda\) equal to the mean of \(\mathrm{diag}(\hat{\mathbf{H}}_l)\) before inversion, following GPTQ practice [2504.09629].

## 4. Integration into the PTQ pipeline

QuEPT is applied at the level of individual **linear operations**, not whole Transformer blocks. It therefore covers attention and MLP linear layers within each block and extends naturally to any sequential layer-wise PTQ pipeline [2504.09629].

The procedure is organized in two stages. First, a full-precision calibration pass collects \(\mathbf{X}_l\) for each layer by forwarding calibration data through layers \(1,\ldots,l-1\) in full precision. Second, quantization proceeds sequentially from input to output. At each layer \(l\), the method computes quantized upstream inputs \(\hat{\mathbf{X}}_l\), forms the upstream discrepancy \(\mathbf{\delta}_l=\mathbf{X}_l-\hat{\mathbf{X}}_l\), computes \(\hat{\mathbf{H}}_l=\hat{\mathbf{X}}_l\hat{\mathbf{X}}_l^\top\), applies damping if necessary, constructs \(\mathbf{W}_l^\ast(\alpha_l)\), and solves

\[
\min_{\hat{\mathbf{W}}_l\in\mathbb{Q}^{n_l\times d_l}}
\left\| \mathbf{W}_l^{\ast}(\alpha_l)\,\hat{\mathbf{X}}_l - \hat{\mathbf{W}}_l\,\hat{\mathbf{X}}_l \right\|_F^2
\]

using a standard PTQ solver [2504.09629].

The solver-specific interpretation is deliberately minimal. In RTN, one applies round-to-nearest directly to \(\mathbf{W}_l^\ast(\alpha_l)\). In GPTQ, one reuses \(\hat{\mathbf{H}}_l\) and its inverse, performs row-wise quantization with residual correction, but targets \(\mathbf{W}_l^\ast(\alpha_l)\). In AWQ, one computes saliency or rescaling as usual, but on \(\mathbf{W}_l^\ast(\alpha_l)\) rather than \(\mathbf{W}_l\). Rotation-based methods such as QuIP, QuaRot, and SpinQuant are treated as orthogonal: QuEPT corrects propagated upstream errors, whereas those methods homogenize weight magnitudes through orthogonal transformations [2504.09629].

The additional computation beyond ordinary PTQ is dominated by forming the correction term \(\mathbf{W}_l\,\mathbf{\delta}_l\,\hat{\mathbf{X}}_l^\top\,\hat{\mathbf{H}}_l^{-1}\). Because \(\hat{\mathbf{H}}_l^{-1}\) is typically already available in Hessian-based PTQ pipelines, the principal new cost is \(\mathbf{\delta}_l\hat{\mathbf{X}}_l^\top\), which the paper characterizes as negligible at scale in practice. Setting \(\alpha_l=0\) on selected layers removes this overhead there entirely [2504.09629].

## 5. Experimental behavior and performance profile

The reported evaluation covers LLaMA2-7B, 13B, and 70B. Calibration uses **128 random 2048-token segments from C4**. Main metrics are WikiText2 perplexity, with PTB and C4 in the appendix, and zero-shot accuracy averaged over ARC Easy, PiQA, and StoryCloze [2504.09629].

The strongest pattern is that QuEPT is most effective in very low-bit settings. Representative WikiText2 perplexity results are summarized below.

| Setting | Baseline \(\rightarrow\) QuEPT | Metric |
|---|---:|---:|
| LLaMA2-7B, RTN, INT2g32 | 90.692 \(\rightarrow\) 12.249 | PPL |
| LLaMA2-7B, RTN, INT2g128 | 4270.828 \(\rightarrow\) 35.291 | PPL |
| LLaMA2-70B, RTN, INT2g128 | 27.268 \(\rightarrow\) 8.799 | PPL |
| LLaMA2-7B, GPTQ, INT3 | 10.881 \(\rightarrow\) 7.898 | PPL |
| LLaMA2-70B, AWQ, INT3g128 | 3.740 \(\rightarrow\) 3.724 | PPL |

The paper also reports INT4 refinements, including LLaMA2-7B GPTQ INT4 from 6.083 to 5.933 and AWQ INT4 from 5.831 to 5.756. Group-wise INT3g128 and INT4g128 remain strong baselines, with QuEPT still improving or matching the best reported values. This suggests that the framework is not limited to rescuing weak baselines; it can refine already competitive PTQ settings [2504.09629].

Zero-shot averages improve in the same general direction. For INT2g128 with RTN, LLaMA2-7B rises from 0.4296 to 0.5598, 13B from 0.4834 to 0.5875, and 70B from 0.5593 to 0.6470. For GPTQ INT3, the corresponding averages increase from 0.6367 to 0.6549 on 7B, from 0.6747 to 0.6853 on 13B, and from 0.7043 to 0.7078 on 70B [2504.09629].

Not every reported cell is monotone. One GPTQ INT2 entry for LLaMA2-13B increases WikiText2 perplexity from 1301.395 to 2782.3528, whereas the 7B and 70B GPTQ INT2 entries improve substantially. This suggests that the empirical advantage is strong overall but not uniform across every model-solver-bit combination [2504.09629].

Runtime measurements on a single NVIDIA V100 GPU indicate modest or negligible overhead. Full-model quantization times are reported as 14.9 minutes, 26.4 minutes, and 2.9 hours for GPTQ on 7B, 13B, and 70B; 13.6 minutes, 25.4 minutes, and 2.4 hours for AWQ; and 10.9 minutes, 19.6 minutes, and 1.7 hours for QEP + RTN. The paper attributes this to reuse of calibration computations [2504.09629].

Robustness experiments on LLaMA2-7B INT3g128 further indicate reduced sensitivity to calibration data. Relative to RTN, GPTQ changes WikiText2 perplexity by \(-0.25\) on C4, \(+0.07\) on PTB, and \(-0.46\) on WT2, whereas QEP+RTN changes it by \(-0.33\), \(-0.30\), and \(-0.49\), respectively. The PTB degradation observed under GPTQ does not appear in the QuEPT-enhanced variant [2504.09629].

## 6. Practical usage, scope, limitations, and nomenclature

The practical guidance in the paper is explicit. For low-bit regimes, especially 2–3 bits, group-wise quantization such as INT2g32, INT2g64, or INT2g128 is preferred, and QuEPT is presented as particularly impactful there. The recommended default is \(\alpha_l=1\) for attention layers and early blocks, with \(\alpha_l \in [0,0.5]\) or \(\alpha_l=0\) for selected MLP blocks to reduce compute and mitigate overfitting. The damping parameter is set to \(\lambda=\mathrm{mean}(\mathrm{diag}(\hat{\mathbf{H}}_l))\). The same calibration data should be used for QuEPT and baseline PTQ in order to minimize preprocessing differences [2504.09629].

Several limitations are stated directly. QuEPT is **weight-only** and **layer-wise**; it does not address activation quantization or joint multi-layer optimization. Calibration data are assumed to be available, and their quality and domain match still matter even though robustness is improved. Hessian inversion may remain costly for very large input dimensions, although damping and selective \(\alpha_l=0\) mitigate part of that cost. Adaptive, data-aware strategies for choosing \(\alpha_l\) per layer or per block remain open [2504.09629].

The acronym itself is not stable across the literature. In the context of [2504.09629], “QuEPT” refers informally to QEP, or Quantization Error Propagation. Elsewhere on arXiv, however, **QuEPT** can denote the unrelated “Quantized Elastic Precision Transformers with One-Shot Calibration for Multi-Bit Switching” [2602.12609], while **QuEPP** denotes “Quantum Enhanced Pauli Propagation” [2603.14485]. This naming overlap is a source of potential confusion, but the method discussed here is the layer-wise PTQ framework centered on propagated quantization-error compensation [2504.09629].

In its intended sense, QuEPT is best understood as a reparameterization of the layer-wise PTQ target rather than a new standalone quantizer. Its contribution is to expose accumulated upstream discrepancy as an explicit optimization variable, convert that discrepancy into a closed-form corrective weight, and retain the quadratic solver structure required by widely used PTQ methods. That combination explains why its gains are largest where local layer objectives are least adequate: the extremely low-bit regime [2504.09629].

Source: https://www.emergentmind.com/topics/quept