Cluster-Based Affine Transformation (CAT)

• Cluster-based Affine Transformation (CAT) is an error-reduction framework for PTQ that clusters quantized logits and applies specific affine corrections to better align with full-precision outputs.
• The method uses PCA for dimensionality reduction and K-Means clustering to compute local affine parameters, resulting in improved accuracy, especially in ultra-low-bit regimes.
• CAT integrates into PTQ pipelines with minimal overhead, offering performance gains on CNNs and transformers without requiring gradient-based fine-tuning.

Cluster-based Affine Transformation (CAT) is an error-reduction framework for post-training quantization (PTQ) that uses cluster-specific affine parameters to align low-bit quantized outputs with their full-precision counterparts. It was introduced for the setting in which plain affine transformation applies a uniform affine parameter set for all outputs and, in low-bit PTQ, can worsen the results rather than improve them. In the formulation reported in "Cat: Post-training quantization error reduction via cluster-based affine transformation" (Zoljodi et al., 30 Sep 2025), CAT operates on quantized logits collected on a calibration set, clusters them after PCA-based dimensionality reduction, and computes an affine correction for each cluster analytically, without requiring fine-tuning of the model or quantization parameters.

1. Origin and problem setting

CAT was proposed in the context of PTQ, where full-precision values are converted into quantized and compressed data types in order to reduce memory footprint and computational overhead (Zoljodi et al., 30 Sep 2025). The motivating problem is the pronounced accuracy degradation of PTQ under a low-bit quantization regime, especially at 2-bit precision, despite PTQ being more cost-efficient than quantization-aware training.

The central observation behind CAT is that affine transformation is a classical technique for reducing the discrepancy between information processed by a quantized model and information processed by its full-precision counterpart, but a single global affine transformation is inadequate in the low-bit regime. The reported finding is that using plain affine transformation, which applies a uniform affine parameter set for all outputs, worsens the results in low-bit PTQ. CAT addresses this by employing cluster-specific parameters to align low-quantized (LQ) outputs with full-precision (FP) counterparts (Zoljodi et al., 30 Sep 2025).

This design places CAT within a post hoc correction paradigm rather than a retraining paradigm. A plausible implication is that the method targets deployment scenarios where recalibration is feasible but gradient-based adaptation is undesirable.

2. Workflow and mathematical formulation

The reported workflow consists of five steps (Zoljodi et al., 30 Sep 2025). First, a classical PTQ pipeline is applied to obtain a quantized model. Second, on a small calibration set, pairs of quantized logits and FP logits are collected for each input. Third, PCA is applied for dimensionality reduction only for clustering, not for the final transformation. Fourth, the reduced quantized logits are clustered, for example via K-Means, and cluster-specific affine parameters are estimated. Fifth, at inference, a given input or logit is assigned to a cluster and the corresponding affine transformation is applied.

The quantization fitting step uses FP and quantized logits, denoted $z_{FP}(x)$ and $z_{LQ}(x)$, together with a temperature $T$, to define

$p_{FP}(x) := \operatorname{softmax}(z_{FP}(x)/T)$

$$p_{LQ}(x;\Theta) := \operatorname{softmax}(z_{LQ}(x;\Theta)/T).$$

The objective is to find quantization parameters $\Theta$ by minimizing KL-divergence:

$$\mathcal{L}_{\text{KL\_out}} = \frac{1}{|\mathcal{X}|} \sum_{x\in\mathcal{X}} \mathrm{KL}\left(p_{FP}(x) \| p_{LQ}(x) \right)$$

with an additional regularizer

$$\mathcal{L}_{\mathrm{reg}} = \|\delta - \delta_0\|_2^2 + \|z - z_0\|_2^2,$$

leading to the final loss

$$\mathcal{L}_{\mathrm{CAT}} = \mathcal{L}_{\mathrm{KL\_out}} + \lambda_p \mathcal{L}_{\mathrm{reg}}.$$

After this refinement, CAT fits a cluster-wise affine map. Quantized logits are first projected by PCA and partitioned into $K$ clusters using K-Means. For each cluster $z_{LQ}(x)$0, the method computes

$z_{LQ}(x)$1

$z_{LQ}(x)$2

$z_{LQ}(x)$3

The affine parameters are then

$z_{LQ}(x)$4

and the cluster-wise correction is

$z_{LQ}(x)$5

An optional blending coefficient $z_{LQ}(x)$6 yields

$z_{LQ}(x)$7

The paper characterizes the fitting procedure as analytic and closed-form, with no backpropagation and no gradient-based learning required (Zoljodi et al., 30 Sep 2025).

3. Clustering rationale and correction mechanism

The motivation for clustering is the observation that quantized logits “cluster” naturally in feature space (Zoljodi et al., 30 Sep 2025). On that basis, the paper argues that cluster-specific affine corrections can capture local discrepancies between FP and LQ outputs more effectively than a single global transformation.

Within each cluster, CAT matches mean and variance or covariance statistics between LQ and FP logits. This gives the method a local correction mechanism over regions of logit space rather than a single output-space calibration. The reported interpretation is that uniform corrections cannot adapt to complex, nonlinear, input-dependent distortions at ultra-low precision, whereas CAT’s cluster-specific approach allows modeling local, nonlinear quantization artifacts (Zoljodi et al., 30 Sep 2025).

PCA serves only the clustering stage. The affine correction itself is always applied in the original logit space. This distinction is explicit in the formulation and avoids conflating representation compression for cluster assignment with the target of the corrective map. A plausible implication is that PCA is treated as a geometric aid for partitioning the calibration samples rather than as part of the deployed prediction transform.

4. Integration into PTQ pipelines

CAT is presented as a plug-in post-process for PTQ workflows and as a framework integrated with PTQ rather than as a replacement for quantization itself (Zoljodi et al., 30 Sep 2025). The method can be applied on top of any quantized model, regardless of the PTQ approach used, with AdaRound, QDrop, LAPQ, and BRECQ listed as examples. The reported requirement is only a modest calibration set, ranging from a few hundred to a few thousand samples, with 500–1,000 samples reported as sufficient to saturate the accuracy benefit in ablations.

The calibration and inference stages remain lightweight. At inference time, CAT assigns a quantized logit to a cluster and applies the corresponding affine correction. The stated additional inference cost consists of cluster prediction with complexity $z_{LQ}(x)$8 and a vector affine transformation (Zoljodi et al., 30 Sep 2025). No extra compute is added at training because the framework is post-training.

The paper also emphasizes that CAT does not require fine-tuning of the model or quantization parameters and does not require gradient computation. In that sense, CAT occupies a specific niche in PTQ research: a deployment-oriented correction layer that modifies outputs rather than re-optimizing network weights.

5. Empirical behavior and reported results

The experimental evaluation reported for CAT is conducted on ImageNet-1K across diverse architectures and low-quantization settings (Zoljodi et al., 30 Sep 2025). The listed model families include ResNet-18/50, MobileNetV2, RegNetX, MNasX2, and vision transformers such as ViT, Swin, and DeiT. The bit-width settings include W4A4, W2A4, W4A2, and W2A2, with particular emphasis on 2-bit regimes.

Several concrete results are reported. On ResNet-18 in the W2A2 setting, the paper gives the following Top-1 accuracies: no affine, 52.84%; plain affine, 52.32%; CAT, 53.18% (Zoljodi et al., 30 Sep 2025). This is presented as evidence that plain affine can worsen results whereas CAT improves them. On ResNet-50 W2A2, CAT is reported to achieve 58.08% versus 57.03% with a strong baseline, PD-Quant, corresponding to a gain of +1.05%. As a plug-in for strong PTQ baselines, CAT is reported to improve Top-1 by 0.3–1.2% on W2A2 across most architectures and baselines, and up to 3.5% in some extreme or corner cases such as AdaRound W2A4. On transformer models, gains are also reported, including up to +0.4% on DeiT-B W2A2.

The ablation results reported in the same source indicate that small-to-moderate cluster counts, such as 8–64, work best for 2-bit settings, while larger $z_{LQ}(x)$9 can overfit noisy quantized representations. Lower PCA dimension is reported to be better for W2A2, whereas higher dimension is more suitable for higher-precision settings; the paper states that 10–50 dimensions typically work well. For the blending coefficient, the reported sweet spot is $T$0 in ultra-low-bit settings, and full reliance on CAT, $T$1, is reported to be worse than blending (Zoljodi et al., 30 Sep 2025).

6. Computational overhead, scope, and limitations

The method is described as adding only a negligible number of additional parameters (Zoljodi et al., 30 Sep 2025). A concrete example is given for ResNet-18 with 11.6M parameters, $T$2 clusters, and $T$3 classes, where the extra parameters are

$T$4

corresponding to approximately 0.9% overhead versus the base model. The paper notes that cluster centroids and affine parameters per cluster must be stored, so the overhead is not zero even though it is characterized as tiny.

The stated strengths are that CAT is generic, gradient-free, post hoc, compatible with both CNNs and ViTs, and especially effective in ultra-low-bit regimes and for large models (Zoljodi et al., 30 Sep 2025). The stated limitations are equally specific: slight storage and bandwidth overhead relative to pure PTQ, possible overfitting or parameter inflation when the number of clusters is too large, reduced marginal benefit as bit-width increases, and potential unsuitability where even 1% extra RAM is unacceptable.

These limitations clarify the practical scope of the method. CAT is not presented as universally optimal across all quantization regimes; rather, its strongest effect is reported when low-bit quantization induces substantial output distortion.

7. Related terminology and potential ambiguity

The acronym “CAT” is not unique in the arXiv literature. In "A Robust and Reliable Point Cloud Recognition Network Under Rigid Transformation" (Liu et al., 2020), CAT denotes Contour-Aware Transformation, a linear transformation of Cartesian point-cloud coordinates into translation- and rotation-invariant representations within the Self Contour-based Transformation framework. That CAT is defined by

$T$5

with $T$6 constructed from contour-derived axes, and is unrelated to PTQ logit correction. The shared acronym can therefore create a nomenclature-level misunderstanding.

A second nearby but distinct line of work is affine-invariant Bayesian clustering, which develops a Bayesian cluster process invariant to classes of linear transformations and able to cluster data without knowing the number of clusters in advance (Huang et al., 2016). That work concerns invariance of clustering under orthogonal, scaling-coordinate orthogonal, or arbitrary non-singular linear transformations, rather than cluster-wise affine correction of quantized logits. The conceptual overlap lies only in the presence of both clustering and affine structure.

Accordingly, Cluster-based Affine Transformation in the PTQ sense refers specifically to the cluster-wise affine alignment of LQ outputs to FP outputs introduced in 2025 (Zoljodi et al., 30 Sep 2025). A common misconception is to read CAT as a general-purpose affine-invariant representation method; the source formulation is narrower and more specific, targeting quantization error reduction in output space.