Rotation-Assisted Post-Training Quantization

• Rotation-Assisted Post-Training Quantization is a technique that applies rotation transforms to neural network weights or activations to reduce quantization error and enhance hardware efficiency.
• It leverages orthogonal and statistically-incoherent transforms, such as Hadamard and Walsh matrices, to address non-Gaussian distributions and mitigate the impact of outliers in ultra-low-bit contexts.
• Empirical results demonstrate that these methods improve metrics like RMSE and KL divergence, proving effective for large models, including LLMs and diffusion transformers.

Rotation-Assisted Post-Training Quantization (PTQ) encompasses a class of quantization methods in which rotations (orthogonal or statistically-incoherent transforms) are applied to neural network weights or activations prior to quantization, with the objective of minimizing quantization error, improving robustness, or optimizing hardware efficiency. These techniques have proven particularly impactful in ultra-low-bit scenarios and for large models such as LLMs or generative Diffusion Transformers, where uniform quantization is typically suboptimal due to non-Gaussian distributions and heavy tails in parameters.

1. Principles and Motivation

Rotation-assisted PTQ methods are founded on the observation that quantization error is highly dependent on the distribution and orientation of the weight or activation tensors with respect to the quantization grid. When weights or activations are transformed via orthogonal (rotation) matrices, their dynamic range and correlation structure can be homogenized, thus reducing misalignment with the quantization bins and minimizing maximum error. Early works such as QuaRot and subsequent approaches (Choi et al., 2 May 2025) showed that standard Hadamard or learned rotation matrices can reduce the root mean squared error (RMSE) after quantization, but significant challenges remain at very low bit widths (e.g., 2-bit).

Rotation techniques also support "incoherence processing," as in QTIP (Tseng et al., 17 Jun 2024), in which random Hadamard transforms are used to make tensor entries behave like independent Gaussian variables, allowing downstream quantization algorithms (e.g., trellis-coded quantization) to perform closer to theoretical optimal packing densities.

2. Rotation Matrix Design and Optimization

Recent advances address the limitations of naive or global rotations. The "Grouped Sequency-arranged Rotation" (GSR) (Choi et al., 2 May 2025) method demonstrates that a training-free approach is feasible: rows of Sylvester-constructed Hadamard matrices are reordered by sequency (sign flip count), yielding Walsh matrices that have clustered frequency components. This arrangement allows creation of block-diagonal (grouped) rotation matrices: $$R_{\mathrm{GSR}} = \mathrm{diag}(H_{\mathrm{wal}}, H_{\mathrm{wal}}, ..., H_{\mathrm{wal}})$$ where each block $H_{\mathrm{wal}}$ is a Walsh matrix with sequency ordering, providing local incoherence and isolating the impact of outliers within each group. Empirical results show that such grouped rotations outperform both global Hadamard and learned rotation techniques (e.g., QuaRot, SpinQuant, OSTQuant) at challenging bit-levels.

In the context of QTIP (Tseng et al., 17 Jun 2024), rotation (incoherence processing) is essential prior to trellis-coded quantization (TCQ). It allows the weights to be treated as i.i.d. Gaussian, matching the assumptions of high-dimensional source coding, and thus enables TCQ decoding in dimensions $\gg 8$ without the exponential codebook growth seen in vector quantization (VQ).

3. Integration with Quantization Algorithms

Rotation can be used either as a preprocessing step before standard rounding quantization or in conjunction with more sophisticated rounding methods. For instance, PTQTP (Xiao et al., 21 Sep 2025) decomposes the rotated weight matrix into structured trit-planes, $$W \approx \sum_{k=1}^2 \mathrm{diag}(\alpha^{(k)}) T^{(k)}$$, allowing for efficient multiplication-free inference in LLMs at 2x1.58-bit precision. Theoretically, this structured decomposition could further benefit from rotation by aligning the most significant modes of $W$ with the axes of the trit-plane basis prior to progressive approximation.

Likewise, model-preserving adaptive rounding (YAQA) (2505.22988) uses Kronecker-factored Hessian approximations, and explicitly recommends incoherence processing (Hadamard rotation) to lower sharpness in the error bounds for full-model KL minimization. Theoretical results in this regime show that optimal rotation (minimizing incoherence parameters $\mu_O, \mu_I$) gives tighter guarantees on output divergence under quantization.

4. Handling Outliers and Distribution Shift

Several rotation-assisted PTQ algorithms implement adaptive selection of rotation schemes to mitigate the impact of activation outliers. For instance, LRQ-DiT (Yang et al., 5 Aug 2025) introduces an Adaptive Rotation Scheme (ARS) for Diffusion Transformers, using the Frobenius norm-based activation fluctuation metric

$J = \| X \|_F / \sqrt{BNC}$

to choose between lightweight Hadamard rotation for mild outliers, or a channel-wise permutation and dual orthogonal rotation approach for salient channel outliers. This dynamic adaptation preserves quantization robustness without the need for end-to-end retraining.

Reliability studies (Yuan et al., 2023) indicate that rotation-assisted methods may intrinsically balance worst-case class performance by redistributing outlier effects, though rigorous analysis under distribution shift remains an active area of research.

5. Hardware and Efficiency Considerations

Rotation matrices chosen to be hardware-friendly (Hadamard, Walsh, block-diagonal, or permutation-based) are critical to PTQ deployment. For instance, power-of-two quantization methods (such as RAPQ (Yao et al., 2022)) benefit from dynamic grouping and BN-informed reconstruction for hardware efficient bit-shift operations. QTIP leverages computed lookup-free Gaussian codes in tandem with bitshift trellis structures, requiring only a few instructions per weight while retaining quantization quality.

Similarly, blockwise or localized rotations (GSR) allow isolation of quantization errors and outliers, streamlining both the implementation and the deployment pipeline. Multiplication-free schemes (PTQTP) offer the highest computational efficiency, leveraging structured rotations to enable uniform ternary operations.

6. Performance and Empirical Results

Empirical results across domains confirm the efficacy of rotation-assisted PTQ techniques, especially at ultra-low bit widths:

• GSR (Choi et al., 2 May 2025) achieves a perplexity of 11.59 on WikiText-2 for 2-bit quantization, matching or surpassing optimization-based rotation algorithms.
• LRQ-DiT (Yang et al., 5 Aug 2025) maintains high image quality for 3-bit quantized DiT models, outperforming existing baselines on COCO, MJHQ, and sDCI datasets.
• PTQTP (Xiao et al., 21 Sep 2025) delivers $82.4\%$ retention of mathematical reasoning versus $0\%$ for competing low-bit PTQ approaches.
• YAQA (2505.22988) reduces KL divergence by approximately $30\%$ compared to conventional rounding algorithms while preserving downstream task accuracy.

7. Future Directions and Theoretical Guarantees

Recent theoretical analyses (Zhang et al., 6 Aug 2025) quantify the propagation and control of quantization error in iterative and rotation-assisted PTQ schemes, providing $\ell_2$ and $\ell_\infty$ bounds as functions of projected column norms, regularization constants, and rotation parameters. These results pave the way for provably optimal rotation schemes tailored for specific hardware, activation distributions, or worst-case error regimes.

Rotation-assisted PTQ stands as a promising framework for scalable, accurate, and efficient quantization of deep neural networks, particularly in the face of distributional heterogeneity, outlier phenomena, and hardware constraints. Its evolution encompasses principled matrix construction (Walsh, Hadamard, learned rotation), adaptive deployment strategies, and rigorous quantitative bounds, marking a substantive contribution to the field of model compression and efficient inference.