GPTAQ: Asymmetric PTQ for LLMs

• GPTAQ is a post-training quantization method that extends GPTQ by calibrating each layer using full-precision outputs to correct quantization errors.
• It employs an asymmetric calibration strategy comparing FP-flow and quantized flows to combat inter-layer error accumulation and drift.
• Incorporating compensation-aware error (CAE), GPTAQ fine-tunes weight adjustments to significantly improve performance metrics in deep LLMs.

Searching arXiv for papers explicitly using the term “GPTAQ” and closely related references. GPTAQ most commonly denotes a compensation-based post-training quantization method for LLMs that extends GPTQ through asymmetric calibration against full-precision layer outputs. In this usage, GPTAQ retains GPTQ’s column-by-column, second-order weight compensation scheme, but changes the calibration target so that each quantized layer is aligned with the corresponding floating-point forward pass rather than only with the quantized computation path. Subsequent work has further refined this framework by arguing that GPTAQ’s residual-error formulation is incomplete unless it also accounts for drift introduced by compensation inside the current layer, yielding the compensation-aware error (CAE) reformulation (Li et al., 9 Apr 2026). The term is nonetheless polysemous: in qualitative-methods research, “GPTAQ” has also been used to mean GPT-assisted qualitative coding, a distinct usage operationalized by QualiGPT (Zhang et al., 2023).

1. Definition and lineage

GPTAQ belongs to the line of post-training quantization (PTQ) methods developed to compress very large transformer models without retraining. Its immediate precursor is GPTQ, which quantizes GPT/OPT-style models layer-by-layer and column-by-column, then compensates the induced output error in the remaining unquantized weights using approximate second-order information (Frantar et al., 2022). GPTQ’s layer objective is to find a quantized weight matrix whose layer outputs remain close to the original outputs on calibration data:

$$\arg\min_{\widehat{\mathbf{W}}} \left\| \mathbf{W}\mathbf{X} - \widehat{\mathbf{W}}\mathbf{X} \right\|_2^2.$$

Within that framework, GPTQ quantizes one column at a time and uses the Hessian, or inverse Hessian, of the layer input covariance to distribute the error caused by quantizing a given column across the remaining weights. The method became notable because it scaled to models such as OPT-175B and BLOOM-176B, quantized to 3 or 4 bits per weight in only a few GPU hours while preserving accuracy close to FP16 (Frantar et al., 2022).

GPTAQ extends this approach by recognizing that layerwise calibration inputs themselves become biased as quantization errors accumulate across layers. Rather than calibrating only against the quantized computation path, GPTAQ introduces an asymmetric objective that uses both the quantized path and the original floating-point path. This suggests that GPTAQ is best understood not as a replacement for GPTQ’s second-order compensation, but as a refinement of its calibration target (Li et al., 9 Apr 2026).

Method	Calibration target	Core distinction
GPTQ	Quantized computation path	Second-order compensation only
GPTAQ	Full-precision layer output	Asymmetric FP-flow/Quant-flow calibration
GPTQ/GPTAQ with CAE	Original full-precision output at each step	Adds compensation-aware error

2. Asymmetric calibration and the FP-flow/Quant-flow distinction

The central conceptual move in GPTAQ is the introduction of two forward-pass views of the model. In the notation used by the later residual-analysis paper, the floating-point path and quantized path are

$$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$

Here, $\widetilde{\mathbf{X}}^l$ denotes floating-point activations and $\mathbf{X}^l$ denotes quantized activations (Li et al., 9 Apr 2026).

This distinction matters because GPTQ calibrates in a “Quant-flow,” where each layer is optimized using inputs produced by already-quantized previous layers. GPTAQ instead aligns each quantized layer with the output that the corresponding full-precision layer would have produced. The asymmetry lies in using full-precision outputs as the reference while the actual inputs seen during layerwise quantization may already contain quantization distortion (Li et al., 9 Apr 2026).

At the column level, GPTAQ introduces a residual term

$$\mathbf{r} = \mathbf{W}^{(q)}\widetilde{\mathbf{X}}_{q:,:} - \mathbf{W}^{(q)}\mathbf{X}_{q:,:},$$

and adds a correction derived from this residual to the standard GPTQ update. In effect, GPTAQ treats inter-layer error accumulation as a first-class calibration signal rather than as an incidental artifact. A plausible implication is that GPTAQ is particularly well matched to deep LLMs, where small mismatches introduced in earlier layers can bias later-layer calibration if left unmodeled.

3. Residual reformulation and compensation-aware error

The main revision introduced in “Rethinking Residual Errors in Compensation-based LLM Quantization” is the claim that GPTAQ’s high-level idea is correct but that its intra-layer residual formulation is still sub-optimal (Li et al., 9 Apr 2026). According to that analysis, GPTAQ’s column-level objective is exact only at the first quantization step within a layer. After compensation has already modified the weights, the iterative target drifts: later updates align the current quantized state with an output computed from compensated weights, rather than with the original full-precision output of the layer.

The revised objective therefore keeps the quantized layer aligned with the original full-precision layer output at every step. The new residual is written as

$$\mathbf{r}' = \widetilde{\mathbf{Y}}^{(0)} - \mathbf{W}^{(q)}.$$

Its key decomposition is

$$\mathbf{r}' = \underbrace{\mathbf{W}^{(q)}(\widetilde{\mathbf{X}}-\mathbf{X})}_{r1} + \underbrace{(\mathbf{W}^{(0)}-\mathbf{W}^{(q)})\widetilde{\mathbf{X}}}_{r2}.$$

The first term, $r1$, is the familiar GPTAQ-style residual produced by the mismatch between FP-flow and Quant-flow inputs. The second term, $r2$, is the new term introduced by the paper and named the compensation-aware error (CAE). CAE captures the output discrepancy caused by the fact that the layer’s weights have already been altered by prior compensation steps. This is a weight-side drift term rather than an input-side mismatch term (Li et al., 9 Apr 2026).

The paper’s central thesis is that these two terms are complementary rather than redundant. Its ablation study reports that adding the extra term

$$(\mathbf{W}^{(0)}_{:,q}-\mathbf{W}^{(q)}_{:,q})\mathbf{2}_{q,q:}$$

improves GPTQ, and adding it to GPTAQ improves GPTAQ further. This supports the view that GPTAQ’s original residual handles inter-layer bias, whereas CAE addresses intra-layer drift introduced by the compensation process itself (Li et al., 9 Apr 2026).

4. Efficient realization: second-order updates, neuron decomposition, and overhead

GPTAQ inherits the practical machinery that made GPTQ usable on billion-parameter models. GPTQ’s efficiency comes from quantizing columns in a fixed order across rows, reusing inverse-Hessian structure, processing columns in blocks, and employing a Cholesky reformulation for numerical stability (Frantar et al., 2022). In the original GPTQ paper, this reduced runtime from the OBQ-style $$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$0 regime to

$$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$1

which the paper describes as faster by a factor of roughly $$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$2 and often about $$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$3 in practice (Frantar et al., 2022).

The later CAE paper emphasizes that its modification preserves this engineering profile. To make CAE efficient, it inherits GPTAQ’s neuron decomposition trick, which decomposes the residual into per-neuron contributions so that when column $$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$4 is quantized, the update needs only the corresponding residual slice (Li et al., 9 Apr 2026). The two decompositions are

$$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$5

and

$$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$6

where

$$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$7

Operationally, the new method adds one extra precomputed matrix $$\widetilde{\mathbf{X}}^l = F(\mathbf{W}^{l-1},\widetilde{\mathbf{X}}^{l-1}),\qquad \mathbf{X}^l = F(\widehat{\mathbf{W}}^{\,l-1},\mathbf{X}^{l-1}).$$8 relative to GPTAQ, while keeping the lazy blockwise update pattern of GPTQ and GPTAQ. The paper explicitly characterizes the resulting algorithm as “literally GPTAQ plus an additional orange-colored CAE term inside the compensation update” (Li et al., 9 Apr 2026). This is significant because it frames the improvement as a small algorithmic change rather than a wholesale redesign.

5. Empirical behavior across model families and quantization regimes

The revised residual formulation is reported to improve both GPTQ and GPTAQ across multiple LLM families, bit-widths, and quantization settings (Li et al., 9 Apr 2026). In 3-bit weight-only quantization on Llama 2 and Llama 3 models, the gains affect both perplexity and downstream average accuracy. On Llama2-7B, adding CAE to GPTQ reduces C4 perplexity from 13.60 to 8.34 and raises average accuracy from 64.9 to 66.5; adding it to GPTAQ improves C4 perplexity from 8.40 to 8.19 and average accuracy from 66.3 to 66.6. On Llama3.1-8B-Instruct, GPTAQ+Ours improves average accuracy from 70.3 to 72.3 and reduces C4 perplexity from 13.85 to 13.79 (Li et al., 9 Apr 2026).

The gains persist in more aggressive settings. Under 2-bit weight-only quantization with QuaRot on Llama2-13B, QuaRot+GPTAQ+Ours reduces C4 perplexity from 17.2 to 16.5 and raises average accuracy from 51.5 to 51.9. In weight-activation quantization, the same pattern appears: on Llama2-13B, SpinQuant+GPTAQ+Ours improves perplexity from 9.55 to 8.60 and average accuracy from 50.2 to 52.2; on Llama3-8B, QuaRot+GPTAQ+Ours improves average accuracy from 43.6 to 44.9 (Li et al., 9 Apr 2026).

The appendix extends this behavior beyond autoregressive LLMs to Vision Transformers. On DeiT-Small under W4A4, accuracy rises from 73.8 to 74.0; under W2A4, DeiT-Base improves from 61.3 to 62.1 (Li et al., 9 Apr 2026). This suggests that the CAE correction is not narrowly tied to one architecture family, even though the motivating discussion is LLM quantization.

These results sit within the broader PTQ trajectory initiated by GPTQ. GPTQ itself reported quantization of OPT-175B in approximately 4.2 hours and BLOOM-176B in 3.8 hours on a single NVIDIA A100, with 3-bit OPT-175B fitting into an 80GB A100 at about 63 GB for weights plus about 9 GB for the key-value cache for a 2048-token context (Frantar et al., 2022). GPTAQ and CAE-based revisions do not replace those deployment motivations; they refine the calibration procedure inside that same compression regime.

Setting	Baseline	With CAE
Llama2-7B, 3-bit GPTQ	C4 13.60, Acc 64.9	C4 8.34, Acc 66.5
Llama2-7B, 3-bit GPTAQ	C4 8.40, Acc 66.3	C4 8.19, Acc 66.6
Llama3.1-8B-Instruct, GPTAQ	C4 13.85, Acc 70.3	C4 13.79, Acc 72.3
Llama2-13B, 2-bit QuaRot+GPTAQ	C4 17.2, Acc 51.5	C4 16.5, Acc 51.9
Llama2-13B, W2A4KV4 SpinQuant+GPTAQ	C4 9.55, Acc 50.2	C4 8.60, Acc 52.2

6. Scope, limitations, and acronym ambiguity

Several misconceptions recur around GPTAQ. One is to treat it as a wholly separate quantization paradigm. The papers instead present it as an extension of GPTQ’s compensation-based PTQ framework: GPTQ supplies the second-order, columnwise compensation machinery, and GPTAQ modifies the calibration target through FP-flow/Quant-flow asymmetry (Frantar et al., 2022, Li et al., 9 Apr 2026). Another misconception is that GPTAQ’s original residual already fully specifies the correct intra-layer objective. The 2026 revision argues that this is not the case, because compensation itself alters the layer and introduces weight-side drift that must be modeled as CAE (Li et al., 9 Apr 2026).

GPTAQ also inherits some of the practical boundaries of its lineage. GPTQ’s main results do not quantize activations, and the paper notes that current mainstream hardware does not directly support efficient mixed-precision operations such as FP16 × INT4 in a way that fully exploits the theoretical compute reduction (Frantar et al., 2022). The CAE paper does not present its contribution as removing those constraints; rather, it shows that better residual modeling yields more accurate PTQ under the same general computational setting (Li et al., 9 Apr 2026).

Finally, the acronym itself is ambiguous. In qualitative-methods research, GPTAQ has been used as shorthand for GPT-assisted qualitative coding, the broader practice that QualiGPT turns into a structured workflow for thematic analysis (Zhang et al., 2023). That usage is unrelated to compensation-based quantization. Nor should GPTAQ be conflated with application-specific GPT systems such as guideline-based automated question answering for prior authorization (Vatsal et al., 2024) or IQAGPT for CT image quality assessment (Chen et al., 2023). In the LLM systems literature, however, GPTAQ ordinarily refers to the asymmetric, compensation-based PTQ method that extends GPTQ and whose residual formulation has since been refined by the introduction of compensation-aware error (Li et al., 9 Apr 2026).