Fair-GPTQ: Bias-Aware 4-Bit Quantization

• Fair-GPTQ is a bias-aware quantization method for large language models that integrates fairness constraints into the 4-bit compression process.
• It reformulates quantization as a constrained optimization over both reconstruction accuracy and group bias using paired stereotypical and anti-stereotypical inputs.
• Empirical results on OPT and Mistral models show reduced stereotype metrics and maintained zero-shot accuracy (at least 90% of baseline) with moderate performance trade-offs.

Fair-GPTQ is a bias-aware post-training quantization method for LLMs that augments the GPTQ family of second-order quantizers with explicit group-fairness constraints. It uses paired stereotypical and anti-stereotypical calibration inputs, adds a bias-aware term to the quantization objective, and steers the rounding operation toward less-biased text generation for protected groups while retaining the memory and speed benefits of 4-bit quantization. In the formulation introduced by the method, quantization is no longer treated as a purely reconstruction-driven compression step: it becomes a constrained optimization problem over both accuracy and group bias, with empirical results showing reduced unfairness relative to a half-precision model and at least 90% of baseline zero-shot accuracy on the reported benchmarks (Proskurina et al., 18 Sep 2025).

1. Position within GPTQ-based quantization

Fair-GPTQ belongs to the GPTQ lineage of one-shot post-training quantization methods. The original GPTQ framework quantizes large transformer layers by minimizing layer-output reconstruction error with approximate second-order information, enabling quantization of models as large as 175 billion parameters to 3 or 4 bits with negligible accuracy degradation relative to the uncompressed baseline, and reporting end-to-end inference speedups of around $3.25\times$ on NVIDIA A100 and $4.5\times$ on NVIDIA A6000 (Frantar et al., 2022). In that setting, the canonical layerwise objective is

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$

with $\mX$ drawn from a calibration set.

Subsequent work on gradient-based post-training quantization argued that GPTQ-style methods are robust to several design variables, that calibration need not be perfectly in-distribution, and that fairness-aware variants could plausibly be built by adding fairness-specific terms to otherwise robust reconstruction losses, but those discussions remained prospective rather than constituting an explicit bias-aware quantizer (Yvinec et al., 2023). Fair-GPTQ is distinguished by making group fairness part of the quantization objective itself and by presenting a concrete algorithm intended to reduce stereotype-related bias during 4-bit compression (Proskurina et al., 18 Sep 2025).

2. Bias-aware objective and second-order formulation

The central modification in Fair-GPTQ is the replacement of the single-calibration-matrix GPTQ objective by a paired-input objective. Two activation matrices are constructed: $\mX_{0}\in\mathbb{R}^{d\times m}$ for stereotypical inputs and $\mX_{1}\in\mathbb{R}^{d\times m}$ for anti-stereotypical counterfactuals. These paired inputs are identical except for a single token representing a protected attribute, such as a gendered pronoun or a racial or religious descriptor. The quantization objective becomes (Proskurina et al., 18 Sep 2025)

$\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$

where $\alpha > 0$ controls the strength of the fairness regularizer.

The first two terms are ordinary reconstruction losses on the stereotypical and anti-stereotypical inputs. The third term penalizes the norm of the post-quantization representation gap induced by $\Delta\mX=\mX_0-\mX_1$. This makes the method group-aware at calibration time: quantization is discouraged from increasing the model’s internal separation between otherwise matched protected-attribute variants.

Using row-wise vectorization $\vw=\operatorname{vec_r}(\mW)$, $4.5\times$0, and $4.5\times$1, the method derives a second-order expansion with non-zero gradient. The matrix gradient and Hessian are

$4.5\times$2

$4.5\times$3

and in vectorized form

$4.5\times$4

A decisive difference from standard GPTQ and OBQ is that Fair-GPTQ does not assume $4.5\times$5 at the pretrained weights. The fairness signal is encoded precisely in this non-zero gradient.

For a constrained scalar quantization step fixing one flattened weight element $4.5\times$6 to $4.5\times$7, the paper gives the closed-form solution

$4.5\times$8

This decomposes naturally into a global debiasing direction, $4.5\times$9, and a GPTQ-style compensation term for the selected weight. The associated saliency is

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$0

In matrix form, the debiasing update is

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$1

This update is applied before the usual GPTQ quantization loop (Proskurina et al., 18 Sep 2025).

3. Algorithmic structure and quantization procedure

Algorithmically, Fair-GPTQ preserves the block-wise, column-wise structure of GPTQ but inserts fairness-aware preprocessing. It first builds an accuracy Hessian

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$2

a bias Hessian

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$3

and the combined Hessian

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$4

It then applies the debiasing shift

$\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$5

After this fairness-aware update, the method re-enters a standard GPTQ-style quantization pass. A Cholesky factor $\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$6 is computed from $\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$7, weights are quantized in blocks of size $\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$8, and quantization error is propagated through the remaining columns by the usual GPTQ compensation rule. Quantization uses symmetric 4-bit quantization with group size 128. For each group $\mW_c = \underset{\mW'}{\arg\min} \|\mW\mX - \mW' \mX\|_{2}^{2},$9, a scale $\mX$0 is computed and columns are quantized as

$\mX$1

In the reported experiments, Fair-GPTQ is not applied uniformly to every matrix. The fairness-aware update is applied only to the attention output projection and output fully connected matrices in each layer, while other matrices are quantized with standard GPTQ. This selective deployment is part of the method’s empirical trade-off between fairness gains and perplexity or zero-shot accuracy degradation (Proskurina et al., 18 Sep 2025).

4. Fairness definition and evaluation protocol

Fair-GPTQ treats bias as differential model behavior under minimal protected-attribute perturbations. At the representation level, the operational quantity is $\mX$2; at the behavioral level, the evaluation is performed with established stereotype and group-bias benchmarks (Proskurina et al., 18 Sep 2025).

CrowS-Pairs measures the percentage of minimal pairs for which the model prefers the more stereotypical sentence. StereoSet reports both a stereotype preference score and an LMS score, the latter measuring whether the model prefers meaningful completions over unrelated ones. Co-occurrence bias is defined, for gender-occupation prompts, by

$\mX$3

BBQ uses ambiguous and disambiguated contexts and scores disambiguated bias as

$\mX$4

while ambiguous-context bias is

$\mX$5

SoFA evaluates variance in log-perplexity across stereotype probes within a category: $\mX$6

The calibration data for the fairness term come from the StereoSet development split, which provides 4,212 stereotype/anti-stereotype sentence pairs. Those pairs are used to construct $\mX$7 and $\mX$8 blockwise during quantization. Standard language-model evaluation is performed with WikiText-2 perplexity and zero-shot tasks including ARC Easy, PIQA, HellaSwag, and Cloze/StoryCloze (Proskurina et al., 18 Sep 2025).

5. Empirical results, ablations, and trade-offs

The reported model families are OPT, with sizes 350M, 1.3B, 2.7B, 6.7B, and 13B, and Mistral-7B v0.3. Quantization is 4-bit, symmetric, with group size 128 and block size $\mX$9. Relative to half precision and GPTQ baselines, Fair-GPTQ reduces stereotype scores while keeping zero-shot accuracy close to baseline levels (Proskurina et al., 18 Sep 2025).

For OPT-6.7B, the main lower-layer configuration yields substantial fairness gains. FP16 records CrowS-Pairs $\mX_{0}\in\mathbb{R}^{d\times m}$0, StereoSet $\mX_{0}\in\mathbb{R}^{d\times m}$1, and Cooc $\mX_{0}\in\mathbb{R}^{d\times m}$2. GPTQ-SS gives CrowS-Pairs $\mX_{0}\in\mathbb{R}^{d\times m}$3, StereoSet $\mX_{0}\in\mathbb{R}^{d\times m}$4, and Cooc $\mX_{0}\in\mathbb{R}^{d\times m}$5. Fair-GPTQ applied to lower layers gives CrowS-Pairs $\mX_{0}\in\mathbb{R}^{d\times m}$6, StereoSet $\mX_{0}\in\mathbb{R}^{d\times m}$7, and Cooc $\mX_{0}\in\mathbb{R}^{d\times m}$8. On Mistral-7B, FP16 gives CrowS-Pairs $\mX_{0}\in\mathbb{R}^{d\times m}$9 and StereoSet $\mX_{1}\in\mathbb{R}^{d\times m}$0, while the lower-layer Fair-GPTQ configuration gives CrowS-Pairs $\mX_{1}\in\mathbb{R}^{d\times m}$1 and StereoSet approximately $\mX_{1}\in\mathbb{R}^{d\times m}$2. On SoFA for OPT-6.7B, the gender score decreases from $\mX_{1}\in\mathbb{R}^{d\times m}$3 in FP16 to $\mX_{1}\in\mathbb{R}^{d\times m}$4 under the lower-layer Fair-GPTQ setting, and the race score decreases from $\mX_{1}\in\mathbb{R}^{d\times m}$5 to $\mX_{1}\in\mathbb{R}^{d\times m}$6.

BBQ results indicate the largest reductions for nationality and religion in both ambiguous and disambiguated contexts. One reported example is nationality bias, which decreases from $\mX_{1}\in\mathbb{R}^{d\times m}$7 to $\mX_{1}\in\mathbb{R}^{d\times m}$8 in ambiguous settings and from $\mX_{1}\in\mathbb{R}^{d\times m}$9 to $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$0 in disambiguated settings. The paper also reports that Fair-GPTQ reduces unfairness relative to a half-precision model and preserves at least 90% of baseline zero-shot accuracy on the stated benchmarks (Proskurina et al., 18 Sep 2025).

The performance cost is measurable but moderate. For OPT-6.7B, FP16 gives ARC Easy $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$1, PIQA $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$2, HellaSwag $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$3, Cloze $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$4, and WikiText-2 perplexity $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$5. The lower-layer Fair-GPTQ configuration gives ARC Easy $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$6, PIQA $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$7, HellaSwag $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$8, Cloze $\mW_c = \underset{\mW'}{\arg\min} \left( \|\mW \mX_0 - \mW' \mX_0\|_2^2 + \|\mW \mX_1 - \mW' \mX_1\|_2^2 + \alpha \|\mW'(\mX_0 - \mX_1)\|_2^2 \right),$9, and perplexity $\alpha > 0$0. The reported pattern is therefore one of improved bias metrics accompanied by higher perplexity and small, usually sub-10% reductions in zero-shot task scores.

Ablations identify the matrices and layers most implicated in the fairness–accuracy trade-off. For OPT-1.3B, applying Fair-GPTQ only to FC2 yields perplexity $\alpha > 0$1 and CrowS-Pairs $\alpha > 0$2, compared with FP16 perplexity $\alpha > 0$3 and CrowS-Pairs $\alpha > 0$4, while applying it to out-proj plus FC2 yields perplexity $\alpha > 0$5 and CrowS-Pairs $\alpha > 0$6. Applying the method to broader matrix sets, such as FC1+FC2, increases perplexity much more strongly, reaching $\alpha > 0$7. The paper therefore emphasizes attention output and FFN down-projection matrices as the strongest leverage points for bias mitigation under quantization (Proskurina et al., 18 Sep 2025).

Comparison with external debiasing methods shows a differentiated picture. On the race subset for OPT-6.7B, FP16 has CrowS-Pairs $\alpha > 0$8; INLP worsens it to $\alpha > 0$9; SentenceDebias worsens it to $\Delta\mX=\mX_0-\mX_1$0; Self-Debias improves it to $\Delta\mX=\mX_0-\mX_1$1 but reduces LMS to $\Delta\mX=\mX_0-\mX_1$2; Fair-GPTQ in the lower-layer configuration reaches $\Delta\mX=\mX_0-\mX_1$3 with LMS $\Delta\mX=\mX_0-\mX_1$4. On gender, Fair-GPTQ gives more modest gains than INLP or SentenceDebias, but with much smaller language-model degradation than Self-Debias. Runtime overhead is also limited: for OPT-6.7B, GPTQ-SS requires $\Delta\mX=\mX_0-\mX_1$5 minutes and Fair-GPTQ applied to all layers requires $\Delta\mX=\mX_0-\mX_1$6 minutes; for Mistral-7B the corresponding values are $\Delta\mX=\mX_0-\mX_1$7 and $\Delta\mX=\mX_0-\mX_1$8 minutes (Proskurina et al., 18 Sep 2025).

6. Interpretive context, neighboring GPTQ directions, and limitations

Fair-GPTQ occupies one branch of a broader diversification of GPTQ research. Some work uses “fairness” in a systems sense, emphasizing predictable latency and balanced resource sharing for 4-bit GPTQ inference on heterogeneous accelerators, as in Opt4GPTQ’s platform-level optimizations inside vLLM (Zhang et al., 29 Oct 2025). Other work uses “fairness” in a representational or error-allocation sense, such as asymmetric calibration for accumulated layerwise error in GPTAQ, which explicitly models the mismatch between full-precision and quantized activations but is not a group-bias method (Li et al., 3 Apr 2025). Fair-GPTQ is distinct from both: its fairness target is social bias in generated text, instantiated through paired protected-attribute prompts and group-fairness regularization (Proskurina et al., 18 Sep 2025).

The method’s limitations are equally specific. The calibration pairs are short StereoSet sentences, so the fairness term is tuned on short contexts rather than long-form discourse. The evaluation is English-centric, and the reported experiments cover OPT and Mistral rather than the full range of current LLM families. Religion-related bias proves harder to reduce than race-related bias. Stronger debiasing, broader matrix coverage, or larger $\Delta\mX=\mX_0-\mX_1$9 can sharply increase perplexity. These constraints indicate that Fair-GPTQ is best understood as a quantization-time bias-mitigation technique with bounded scope rather than as a complete debiasing solution (Proskurina et al., 18 Sep 2025).

A plausible implication is that Fair-GPTQ establishes quantization as an intervention point for fairness rather than merely a compression stage. Earlier GPTQ work suggested that fairness-aware quantization might be achieved by augmenting reconstruction objectives with group-conditioned terms while preserving robust low-level design choices (Yvinec et al., 2023). Fair-GPTQ realizes that proposal in explicit second-order form. Its significance lies not only in the reported reduction of CrowS-Pairs, StereoSet, BBQ, and SoFA scores, but also in demonstrating that the rounding decisions of a 4-bit quantizer can be guided by social-bias criteria without abandoning the computational structure that made GPTQ practically important in the first place (Proskurina et al., 18 Sep 2025).