QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation

Abstract: Recent training-free post-training quantization methods restore model accuracy through closed-form residual compensation. To constrain additional model storage overhead, several existing methods gate layer selection by goodness-of-fit, retaining only those layers whose compensation yields a positive residual fit score and discarding the rest. In this paper, we show that, under the low-bit W4A4 setting, this gating mechanism fails to distinguish poorly predictable quantization error from numerical solver failure. Rank-deficient input activations yield severely ill-conditioned or numerically singular Gram matrices, causing the closed-form solver to become unstable and produce spuriously negative fit scores. Consequently, existing goodness-of-fit gates misclassify affected layers as uncompensable and discard them. Many of these discarded layers can nevertheless provide substantial error recovery when their compensation is computed using a numerically stable solver. To address this problem, we propose a parameter-free truncated pseudoinverse solver which removes collapsed directions prior to inversion. On ViT-B with the W4A4 setting, our training-free method achieves 81.42\% top-1 accuracy, outperforming prior post-training methods and fine-tuning-based baselines. Combined with joint low-rank and quantization compression, the proposed method reaches a deployable operating point of 80.26\% accuracy at 54.7 MB, providing a well-balanced trade-off between model size and accuracy.

• The paper identifies activation-induced rank deficiency, rather than ill conditioning alone, as the cause of least-squares compensation collapse and erroneous negative-R gating under W4A4 quantization.
• QuaSAR replaces direct Gram-matrix inversion with a parameter-free truncated pseudoinverse, recovering valuable compensators and achieving 81.42% 1 0.06 top-1 accuracy on ViT-B/ImageNet, compared with 68.5% for the RepQ-ViT baseline.
• The paper combines activation-aware low-rank compression with int8/int4 factor quantization to reach 80.26% 1 0.11 accuracy at 54.7 MB, while finding that uniform rank allocation is as effective as importance-based allocation.

QuaSAR addresses a specific failure mode in training-free post-training quantization (PTQ) with residual compensation: under the W4A4 setting, closed-form least-squares compensation can collapse numerically on rank-deficient activations, and the standard goodness-of-fit gating criterion then discards precisely the layers whose compensation is most valuable. The paper's contribution is primarily diagnostic — identifying that rank deficiency, not ill conditioning per se, triggers collapse — together with a parameter-free truncated pseudoinverse solver and a dual-axis compression scheme that yields a deployable accuracy–size operating point.

Background and problem setting

The paper situates itself within the line of work that augments quantized backbones with lightweight structural compensators. QwT appends parallel linear layers to quantized blocks and solves a closed-form least-squares regression on a calibration set to predict the residual quantization error; QwT-v2 replaces these with diagonal affine operators that fold into quantization parameters, achieving near-zero overhead but reduced capacity. Between the full-matrix QwT (~30% parameter footprint) and the diagonal QwT-v2, the authors argue there is no operating point that balances accuracy and size under W4A4 — the setting most relevant for efficient vision inference and substantially more sensitive to degradation than weight-only quantization.

The compensation objective is a least-squares fit of the quantization error $E = Y_{\text{fp}} - Y_{\text{q}}$ as a linear function of the layer input, solved via inversion of the activation Gram matrix $G = \tilde{X}^T\tilde{X}$. Prior work (e.g., GPTQ's diagonal dampening, ridge regularization) treats the well-known ill conditioning of this matrix as an implementation detail, preemptively regularizing without analyzing when and why compensation actually fails.

Diagnosis: rank deficiency triggers collapse, and the gate misclassifies it

The central empirical finding is that certain layers have intrinsically redundant activations whose effective dimensionality is far below nominal width, making $G$ near rank-deficient even at full precision. Activation quantization at W4A4 compresses nearly identical input dimensions further, pushing these layers past the threshold of numerical collapse. The fingerprints are unambiguous: the smallest Gram eigenvalue falls within the float32 noise floor ($\approx \epsilon \cdot \lambda_{\max}$), the compensation parameter norm explodes (e.g., $6.2 \times 10^7$ for ViT-B layer b0.fc2), and the fitted $R^2$ becomes mathematically impossible (negative on the training data itself, e.g., $R^2 = -68.6$).

The consequential error is downstream of the solver: the widely used $R^2 > 0$ gate interprets this numerical collapse as "uncompensable" and discards the layer. The paper's sharpest claim is that this gate discards exactly what it should keep — b0.fc2, the layer most confidently discarded, is also the single most valuable compensator in ViT-B, with its removal costing 1.05% top-1 accuracy, more than any other layer. Notably, a large condition number alone does not predict collapse: the layer with the largest condition number in ViT-B does not collapse, whereas every collapsed layer exhibits a numerically singular Gram matrix. This supports the paper's claim that rank deficiency, not ill conditioning per se, is the trigger.

The truncated pseudoinverse solver

The fix is deliberately minimal: replace naive inversion with a truncated pseudoinverse that zeroes out eigenvalues below a threshold $\tau = \text{rcond} \cdot \lambda_{\max}$ (set to $10^{-3}$) and takes reciprocals only of surviving directions. The authors argue, and empirically verify, that discarded directions contain no reliable information, so truncation eliminates numerical explosion without compromising effective compensation. A useful theoretical property is preserved: because the intercept direction of the augmented input has an eigenvalue scaling with sample count, it always survives truncation, guaranteeing $G = \tilde{X}^T\tilde{X}$0 by construction.

Critically, the paper disentangles stability from shrinkage. Ridge regularization achieves 81.64% ± 0.04 versus 81.42% ± 0.06 for the truncated pseudoinverse on ViT-B/W4A4 — statistically indistinguishable — demonstrating that the gain comes from discarding collapsed directions rather than from the biased universal shrinkage ridge applies to all directions. Direct inversion, by contrast, collapses in 3 of 5 seeds on ViT-B, with collapsed seeds dropping to near-random accuracy (~0.1%); its reported 79.52% is a fortunate average over uncollapsed seeds.

Layer-wise framework and results

The framework attaches an independent compensator to each of the four linear layers within a Transformer block rather than sharing one per block (a +3.1% gain over block-wise compensation), and solves sequentially: each compensated layer is plugged back into the model before capturing inputs for the next layer, ensuring the regression sees the true deployment-time input distribution.

On ViT-B/W4A4/ImageNet with RepQ-ViT as the baseline quantizer (68.5% top-1), the full method achieves 81.42% ± 0.06 training-free, exceeding training-free QwT (76.3%), QwT-v2 (75.6%), IGQ-ViT (79.3%), and the fine-tuning-based QwT* (78.5%). The uncompressed full-matrix compensator reaches the same accuracy but at 215 MB; after compression it retains 80.26% ± 0.11 at 54.7 MB — dominating QwT* by 1.76% at 4.4 MB smaller. Cross-architecture experiments on DeiT-T, Swin-T, and ResNet-50 confirm that ill conditioning is universal (condition numbers up to $G = \tilde{X}^T\tilde{X}$1, with ResNet-50's Gram matrix exactly singular), but that failure modes differ: silent collapse (ViT-B), high seed variance (Swin-T), gated-layer discarding (DeiT-T), and outright solver exception (ResNet-50). Numerical stabilization yields consistent gains across all four, though the authors are careful to state this validates the stabilization mechanism, not optimality of the full framework on every architecture.

Compression axes and the allocation finding

The compensator is compressed along two orthogonal axes. First, activation-aware low-rank decomposition whitens the Gram matrix before truncated SVD, minimizing output-space error $G = \tilde{X}^T\tilde{X}$2 rather than parameter-space error, so retained rank concentrates in directions activations actually traverse; the whitening reuses the same truncation threshold, remaining well-defined even for singular layers. Second, channel-wise symmetric int8/int4 quantization is applied to the low-rank factors, with fp16 scales and full-precision bias.

An ablation on rank allocation under a fixed 54.7 MB budget yields a counterintuitive result: uniform allocation (80.376%) matches or beats importance-weighted allocations, and exactly reversing a high-rank-to-important-layers scheme changes accuracy by only 0.02%. The paper concludes that layer importance does not predict rank sensitivity, and that compensation budgets should be allocated at global scale rather than position-wise. Under fixed size, higher rank with more aggressive factor quantization outperforms lower rank with milder quantization, indicating compensation benefits more from retained subspace dimensionality than per-parameter precision.

Limitations and open questions

The paper is candid about several boundaries. The complete layer-wise framework does not yet stably outperform the block-level baseline on CNNs — module-level compensation on ResNet-50 remains an open challenge attributed to convolutional error-propagation structure, even though the stable solver itself works there. At lower bit widths such as W3A3, the baseline quantizer degrades so severely that stabilizing solver and quantizer simultaneously is unresolved. The rcond threshold is set empirically to $G = \tilde{X}^T\tilde{X}$3 without a principled selection rule, and the solver is validated only with RepQ-ViT as the quantizer; combining it with other W4A4 quantizers is proposed but untested. The cross-architecture claim is explicitly scoped to the universality of numerical stabilization rather than end-to-end framework superiority.

Conclusion

QuaSAR's main contribution is a field-wide diagnosis: closed-form residual compensation under W4A4 fails through a mechanism — activation-quantization-induced rank collapse followed by erroneous gating on negative $G = \tilde{X}^T\tilde{X}$4 — that prior methods obscured by treating solver instability as a mere implementation detail. A parameter-free truncated pseudoinverse recovers the discarded high-value layers, and the resulting framework establishes a strong training-free operating point (81.42% on ViT-B/W4A4; 80.26% at 54.7 MB compressed) that dominates both QwT variants and fine-tuning-based QwT* in accuracy and size. The uniform-allocation finding further simplifies deployment by removing the need for importance-based budgeting. The remaining open questions — CNN module-level compensation, sub-W4 bit widths, and quantizer-agnostic validation — are concrete and well-scoped.