---
title: 'GPTQ: Efficient Transformer Weight Quantization'
url: https://www.emergentmind.com/topics/generative-pretrained-transformer-quantization-gptq
type: topic
---

# GPTQ: Efficient Transformer Weight Quantization

Generative Pretrained Transformer Quantization (GPTQ) is a one-shot, post-training weight quantization method for large GPT- and OPT-style Transformers that uses approximate second-order information to minimize layer-output error on fixed calibration data while remaining practical at very large scale [2210.17323]. In subsequent PTQ taxonomies, GPTQ is typically placed in the quantization error mitigation stage: it does not flatten the activation or weight distribution by itself, but instead compensates the loss increase caused by quantization through Hessian-aware, block-wise feedback updates to not-yet-quantized weights [2507.17417].

## 1. Formal objective and conceptual role

For a linear layer with inputs $X \in \mathbb{R}^{n \times d}$, weights $W \in \mathbb{R}^{d \times m}$, and output $Y = XW$, GPTQ quantizes $W$ to $\hat{W}$ by minimizing the layer reconstruction error on calibration activations:
$$
\min_{\hat{W} \in \mathcal{Q}_b} \;\| X (W - \hat{W}) \|_F^2.
$$
For a single weight vector, the associated second-order form is
$$
\Delta L \approx \tfrac{1}{2}\,\Delta w^\top H\,\Delta w,\quad H = X^\top X,
$$
and later reviews present the same idea as minimizing the quantization-induced loss
$$
\epsilon = \mathcal{L}(W_q)-\mathcal{L}(W)=\mathcal{L}(W+\delta)-\mathcal{L}(W)\approx \tfrac{1}{2}\delta^\top H(W)\delta,
$$
with $\delta = W_q - W$ and $H(W)$ estimated from calibration data [2210.17323; 2507.17417].

This formulation gives GPTQ a specific position among PTQ methods. Pre-quantization transformations such as shifting, scaling, and rotation attempt to make data distributions flatter or less outlier-dominated before quantization. GPTQ, by contrast, is the compensatory stage that operates after such preprocessing decisions, using curvature information to choose rounding decisions and propagate their effect across remaining variables [2507.17417].

A common uniform quantizer used in GPTQ discussions is
$$
X_q = \text{round}\left(\frac{X}{\Delta}\right),\quad
\Delta = \frac{\max(|X|)}{2^N - 1},
$$
with dequantization
$$
\text{dequant}(X)=X_q\Delta.
$$
The original GPTQ paper also states a more general affine form with scale $s$ and optional zero-point $z$:
$$
q = \operatorname{clip}(\operatorname{round}((w - z)/s),\; q_{\min},\; q_{\max}), \quad \hat{w} = s\,q + z.
$$
These formulations describe the discrete grid over which GPTQ solves its reconstruction problem [2507.17417; 2210.17323].

## 2. Greedy second-order quantization procedure

GPTQ is a greedy, one-shot, second-order procedure that quantizes a layer one coordinate at a time while compensating the remaining full-precision coordinates. In the original presentation, for a variable $i$ with scalar quantization error $e_i = \hat{w}_i - w_i$, the compensation applied to the remaining free variables is
$$
\delta_F = -\frac{e_i}{(H_F^{-1})_{ii}}(H_F^{-1})_{:,i},
$$
and for a set $Q$ quantized together the corresponding block update uses the Schur complement:
$$
\delta_F = -\big(w_Q - \hat{w}_Q\big)\big([H_F^{-1}]_{QQ}\big)^{-1}(H_F^{-1})_{:,Q}.
$$
To keep these updates numerically stable at LLM scale, GPTQ uses a Cholesky-based treatment of $H^{-1}$ together with mild diagonal damping [2210.17323].

The practical workflow described in later evaluations is layer-wise. Calibration activations are collected, an approximate Hessian is estimated from them, the Hessian is factorized with LDL or Cholesky structure, and weights are partitioned into blocks or groups. Columns inside each block are then processed sequentially: a scale is chosen, the current column is quantized, the resulting error is computed, and feedback compensation is applied to the not-yet-quantized columns. In a representative fair-comparison setup, GPTQ calibration used 128 sequences from WikiText-2 with length 2048, and the main experiments applied GPTQ to transformer linear layers under W4A4 or W4A16 settings [2507.17417].

The original implementation emphasized scale as well as accuracy. GPTQ quantized OPT-175B and BLOOM-176B in approximately four GPU hours on a single NVIDIA A100 80GB, and its blockwise formulation was designed to reduce memory traffic during autoregressive generation. The same paper reports that fixed column order performs similarly to greedy order at GPT scale, while block size $B \approx 128$ balances throughput and stability [2210.17323].

## 3. Calibration, granularity, and interaction with preprocessing

GPTQ is fundamentally a weight-quantization method. In the original paper, the large linear layers of attention and MLP/FFN modules are quantized, while embeddings and the final LM head are kept in FP16 in the 175B single-GPU configuration; LayerNorms and biases remain in floating point [2210.17323]. Later systematizations treat activation quantization as a separate design choice layered around GPTQ rather than as part of GPTQ itself [2507.17417].

Granularity is central to GPTQ’s behavior. The original method uses per-row asymmetric min-max grids by default and also supports group-wise quantization, which becomes particularly important at 2–3 bits. In a later fair evaluation, GPTQ was studied with per-channel or per-group quantization, with group size 128 in the main experiments. That study also reports a direct storage trade-off for symmetric weight quantization with FP16 scales: the average extra bits per value are 0.5 for group-32, 0.25 for group-64, 0.125 for group-128, 0.0625 for group-256, and 0.03125 for group-512 [2210.17323; 2507.17417].

GPTQ also interacts strongly with pre-quantization transformations. Rotation and scaling do not belong to GPTQ itself, but they change the activation and weight distributions that GPTQ sees. In a decoupled INT4/W4A4 study, optimized rotation plus scaling paired with GPTQ consistently outperformed rotation alone and yielded the strongest results; with non-optimized transforms, rotation alone could be better than rotation plus scaling; and for MXFP4 the optimal INT4 pre-transform strategy did not generalize, with scaling-based approaches becoming more effective than rotation-centric ones [2507.17417].

These results support a precise interpretation of GPTQ’s role. GPTQ is not primarily an outlier suppression method. Outlier handling occurs upstream through scaling, rotation, or related transforms; GPTQ then compensates the residual quantization error left after those transformations [2507.17417].

## 4. Empirical performance and operating regimes

The original GPTQ paper established the method’s reputation through 3-bit and 4-bit weight-only quantization at very large scale. For OPT-175B, the FP16 baseline achieved WikiText2 perplexity 8.34, PTB 10.33, C4 9.56, and LAMBADA 75.59; GPTQ 4-bit reported 8.37, 10.47, 9.67, and 76.80 respectively. GPTQ 3-bit reported 8.68, 10.92, 9.98, and 76.19, and with group size 1024 it reached 8.45, 10.69, 9.84, and 77.39. The same study reported latency per token on OPT-175B of 230 ms in FP16 versus 121 ms with GPTQ 3-bit on a single A100 80GB, a 1.90× speedup, and 589 ms versus 132 ms on A6000, a 4.46× speedup; 3-bit OPT-175B weights occupied approximately 63 GB, with approximately 9 GB KV cache for 2048 tokens [2210.17323].

Subsequent evaluations refined the empirical picture. In a controlled W4A4 benchmark on LLaMA-3.2-1B, optimized rotation plus scaling plus GPTQ achieved WikiText-2 perplexity 11.80 and 0-shot average 48.89 in per-channel mode, while adding a low-rank branch produced 11.73 and 49.06. In per-group-128 mode, GPTQ reported 10.89 perplexity and 50.63 0-shot average, and GPTQ plus low-rank reported 10.87 and 51.83. The same study concluded that GPTQ is the primary driver of error mitigation, with low-rank branches occasionally providing additional gains [2507.17417].

Task-specific studies show greater variability. One 4-bit comparison on LLaMA 3.2–1B, Qwen 0.5B, and Phi 1.5B reported that GPTQ outperformed GSQ in accuracy for MS MARCO and BoolQ, especially for LLaMA 1B, where the reported best quantized accuracies were 99.86% on MS MARCO versus an 81.12% baseline and 62.17% on BoolQ versus a 40.15% baseline. The same study also states that GPTQ achieves very low accuracy on GSM8K across all models, reports a 0.00% GPTQ result for Qwen 0.5B on GSM8K, and notes a “significant drop in throughput across all datasets” for GPTQ together with several cases of increased memory usage labeled a “Memory Paradox” and attributed to “experimental issues or inefficient implementation” [2508.11318].

A different line of evaluation emphasized distribution mismatch. For LLaMA-1B under INT4 fine-grain quantization, GPTQ calibrated on C4 produced C4 perplexity 9.34, close to the FP16 baseline 8.91, but PTB-new 1396.76 and PTB 1399.89 versus FP16 values 58.34 and 37.39. The same study concluded that GPTQ can overfit the calibration distribution, and that its benefits in zero-shot accuracy can be limited when activations remain FP16 [2312.08583].

## 5. Variants, descendants, and theoretical reinterpretations

GPTQ has become a reference point rather than a fixed endpoint. One line of work modifies the calibration objective. GPTAQ introduces asymmetric calibration, always matching the quantized layer’s output to the exact output in the full-precision model, and derives a closed-form update
$$
\Delta w = (w_q - \hat w_q)\frac{(H^{-1})_{:,q}}{(H^{-1})_{qq}} + r X^\top H^{-1},
$$
where the second term explicitly corrects accumulated asymmetry error. That paper reports that GPTAQ requires around 20 more lines than GPTQ and quantized LLaMA3.1-405B and EVA-02 on one A100 [2504.02692]. ResComp further argues that the residual error originates not only from the output difference of the preceding layer but also from the discrepancy between compensated and original weights within each layer, introducing a “compensation-aware error” term and showing consistent gains over GPTQ and GPTAQ across low-bit settings [2604.07955].

A second direction replaces or augments GPTQ’s solver. CDQuant optimizes the same layer-wise quadratic reconstruction objective but uses greedy coordinate descent rather than one-pass second-order compensation, and reports consistent improvements over GPTQ in 2–4 bit weight quantization on PaLM2 and Gemma families; for example, on PaLM2-Otter INT3 per-channel, BCD reported perplexity 6.979 versus 7.176 for GPTQ, and on INT2 group-$128$ it reported 9.719–9.760 versus 10.816 [2406.17542]. Other work augments GPTQ with low-rank structure more directly: GPTQ-intrinsic LoRA incorporates the low-rank correction into the GPTQ pass through an augmented Hessian, proving layer-wise bounds in which the usual GPTQ dependence on $\|X\|_F^2$ is replaced by the rank-$r$ residual $\|X-X_r\|_F^2$, and reports substantial perplexity improvements on Qwen3 language models at 3–4 bits [2606.01412].

A third direction reinterprets GPTQ theoretically. One paper proves that, when executed back-to-front for a linear layer, GPTQ is mathematically identical to Babai’s nearest plane algorithm for the closest vector problem on a lattice defined by the Hessian of the layer’s inputs. Under the no-clipping condition, GPTQ therefore inherits Babai’s error upper bound, and the familiar error propagation step acquires an explicit geometric meaning as projection onto successive nearest planes [2507.18553].

GPTQ has also been adapted to specialized objectives. Fair-GPTQ adds a group-bias term to the quantization objective using paired activations $X_0$ and $X_1$:
$$
W_c = \arg\min_{W'} \Big(\|W X_0 - W' X_0\|_2^2 + \|W X_1 - W' X_1\|_2^2 + \alpha \|W'(X_0-X_1)\|_2^2\Big),
$$
and reports reduced unfairness relative to both FP16 and GPTQ baselines while preserving at least 90% of baseline zero-shot accuracy and retaining the speed and memory benefits of 4-bit quantization [2509.15206]. In video diffusion, timestep-aware SVDQuant-GPTQ combines low-rank outlier absorption with GPTQ-based residual quantization and expert- and timestep-specific activation calibration, reducing peak GPU memory by 59.3% relative to BF16 with only a 0.9% drop in VBench average score on OpenS2V-Eval [2605.27003].

## 6. Limitations, failure modes, and practical status

Several limitations recur across the literature. GPTQ depends on calibration activations, so calibration mismatch can materially affect quality; this is explicit in the original paper and is dramatized by the severe PTB degradation reported under mismatched LLaMA-1B calibration [2210.17323; 2312.08583]. Certain matrices can be more sensitive than others, and group-wise scales or mixed precision remain standard mitigations for high-curvature or outlier-heavy layers [2210.17323].

Numerical stability is another persistent issue. GPTQ relies on curvature estimation and matrix factorization, so damping and Cholesky- or LDL-based formulations are essential at scale. The original paper states that 4-bit is generally safe, 3-bit is often excellent, especially with grouping, and 2-bit requires small groups to remain reasonable; later W4A4 evaluations similarly conclude that W4A4 remains challenging for small LLMs and that non-optimized transforms can underperform substantially [2210.17323; 2507.17417].

Implementation outcomes can diverge from algorithmic expectations. Although GPTQ is usually motivated by reduced memory movement and better inference efficiency, one application study reported minimal latency benefits, a significant drop in throughput across all datasets for GPTQ, and multiple cases in which the quantized model used more memory than the baseline. The authors labeled these anomalies a “Memory Paradox” and attributed them to “experimental issues or inefficient implementation,” underscoring that GPTQ’s theoretical compression benefits do not automatically imply favorable end-to-end systems behavior [2508.11318].

The current practical consensus is therefore conditional rather than absolute. GPTQ remains one of the standard methods for one-shot PTQ at LLM scale because it combines a well-defined Hessian-aware reconstruction objective, a scalable greedy solver, and strong empirical performance at 3–4 bits on many transformer layers [2210.17323; 2507.18553]. At the same time, recent work treats it as a modular core that benefits from better preprocessing, alternative compensation schemes, low-rank augmentation, asymmetric calibration, or fairness-aware objectives rather than as a complete solution in isolation [2507.17417; 2504.02692; 2604.07955; 2606.01412; 2509.15206].

Source: https://www.emergentmind.com/topics/generative-pretrained-transformer-quantization-gptq