OSAQ: Outlier Self-Absorption Quantization

• Outlier Self-Absorption Quantization (OSAQ) is a technique that adds controlled perturbations in the stable, low-curvature Hessian subspace to suppress extreme weight outliers.
• It uses a closed-form additive solution with a softmax-weighted quadratic approximation to efficiently minimize the range-dominating effects of large-magnitude weights.
• OSAQ integrates as a plug-and-play post-training step, enhancing low-bit quantization performance—especially in 2-bit regimes—without incurring inference-time overhead.

Outlier Self-Absorption Quantization (OSAQ) is a post-training technique for low-bit weight-only quantization of LLMs that targets systematic large-magnitude weight outliers through an additive transformation guided by second-order Hessian structure. Its central premise is that certain weight perturbations can be added directly to a layer without materially changing task loss because those perturbations lie in a stable low-curvature subspace of the Hessian. OSAQ therefore addresses low-bit quantization through additive weight suppression rather than the more common multiplicative families based on scaling or rotation, and it is designed to be absorbed offline into the weight tensor with no inference-time overhead (Li et al., 6 May 2026).

1. Quantization problem and additive formulation

OSAQ is motivated by the standard uniform quantization pipeline for a floating-point weight $w$: $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$ with

$$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$

In this formulation, a small number of large-magnitude weights enlarge $w_{\max}-w_{\min}$, increase the scale $s$, and coarsen the quantization grid for the majority of weights. The effect is especially severe in $2$-bit, $3$-bit, and $4$-bit regimes, where only $4$, $8$, or $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$0 levels are available (Li et al., 6 May 2026).

The method is positioned against two major prior patterns. One is quantization-error compensation, exemplified by GPTQ. The other is equivalent multiplicative transformation between adjacent layers: $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$1 where $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$2 is either a scaling vector or a rotation matrix. In the OSAQ framing, such multiplicative methods can help, but low-bit performance remains unsatisfactory, especially at $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$3–$$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$4 bits, and they require inter-layer coupling. OSAQ instead seeks an offline additive transformation fully absorbed into the weight tensor itself, with no inter-layer bookkeeping and no inference-time cost (Li et al., 6 May 2026).

The term “self-absorption” refers to this layer-local mechanism: the layer absorbs its own outliers without needing a compensating transformation elsewhere. This is not exact functional equivalence in the multiplicative sense. It is a loss-preserving perturbation to second order.

2. Second-order basis and stable null space

OSAQ is derived from a second-order approximation of task loss around a flattened layer weight vector $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$5. For a perturbation $$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$6,

$$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$7

where

$$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$8

The approximation reflects the paper’s view that the dominant practical criterion is low curvature under the Hessian (Li et al., 6 May 2026).

Two empirical observations are then used. First, the Hessian with respect to a layer’s weights is strongly low-rank. Second, this low-rank structure is claimed to be consistent across different input samples. In particular, some directions repeatedly exhibit essentially zero curvature under different calibration inputs. OSAQ treats these directions as a stable null space. If a perturbation lies inside this subspace, then the second-order loss term is approximately zero.

With eigendecomposition

$$w^{(Z)}=\operatorname{clip}\left(\left[\frac{w}{s}\right]+z,\;0,\;2^b-1\right), \qquad \hat w=s\left(w^{(Z)}-z\right)\approx w,$$9

OSAQ defines the practical null space by selecting the smallest-eigenvalue tail through a tail-energy rule: $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$0 Here $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$1 denotes the null-space basis matrix. The paper is explicit that this “null space” is practical rather than exact: it is formed by vanishing-curvature directions selected by a tail-energy heuristic, not by exact zero eigenvalues (Li et al., 6 May 2026).

This construction suggests that OSAQ is less about discovering a new quantizer than about exploiting a reusable low-curvature subspace in which weights can be moved to reduce outliers while preserving loss.

3. Additive self-absorption mechanism

For a layer weight matrix $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$2, OSAQ parameterizes the perturbation as

$$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$3

Each output channel receives its own linear combination of the $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$4 null-space directions. The transformed weight is

$$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$5

The direct objective would be to minimize the infinity norm,

$$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$6

because the quantization bottleneck is driven by range-dominating outliers. To avoid iterative optimization, OSAQ replaces this with a softmax-weighted quadratic proxy (Li et al., 6 May 2026).

For output channel $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$7, define

$$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$8

where $$s=\frac{w_{\max}-w_{\min}}{2^b-1}, \qquad z=\left[\frac{0-w_{\min}}{s}\right].$$9 is a temperature. As $w_{\max}-w_{\min}$0, the weighting concentrates on the largest-magnitude entries. The per-channel optimization becomes

$w_{\max}-w_{\min}$1

with

$w_{\max}-w_{\min}$2

The first term suppresses large entries, the second regularizes the coefficients, and the third is an anti-shift constraint that penalizes uniform channel translation.

A key feature of OSAQ is that this objective has a closed-form solution. Setting derivatives to zero yields

$w_{\max}-w_{\min}$3

where

$w_{\max}-w_{\min}$4

Hence

$w_{\max}-w_{\min}$5

Because $w_{\max}-w_{\min}$6, each $w_{\max}-w_{\min}$7 is symmetric positive definite and invertible, so $w_{\max}-w_{\min}$8 is the unique global minimizer (Li et al., 6 May 2026).

This closed-form construction is the technical core of OSAQ. Outlier suppression is achieved by moving each row in low-curvature directions that are cheap in loss but effective in reducing range.

4. Integration into post-training quantization

OSAQ is a plug-and-play preprocessing step applied before a downstream quantizer such as GPTQ, AWQ, QuIP, or even activation-quantization methods. When combined with another method, OSAQ is always applied first. The layerwise workflow is: estimate the approximate Hessian, eigendecompose it, select the stable null-space dimension $w_{\max}-w_{\min}$9 with the tail-energy threshold $s$0, compute the softmax weights $s$1, solve for $s$2, update $s$3, and then run the chosen PTQ algorithm on the transformed weights (Li et al., 6 May 2026).

For GPTQ integration, the paper follows the GPTQ setup of 128 samples with sequence length 2048. The method requires calibration data of the standard PTQ type, but no retraining, no fine-tuning, and no iterative inner loop for the coefficient solve. The additive transformation is absorbed offline into the stored weight tensor.

This offline placement is central to the method’s systems claim. On an Nvidia A100, reported quantization-time costs are modest: for LLaMA2-7B, GPTQ takes 22 min and OSAQ+GPTQ 24 min; for 13B, 40 min versus 45 min; for 70B, 4.0 hr versus 4.6 hr. Inference-time overhead is zero because deployment uses the same quantized model structure after the weight tensor has been rewritten (Li et al., 6 May 2026).

A plausible implication is that OSAQ is most attractive in deployment settings where kernel simplicity and unchanged inference graphs are as important as quantization accuracy.

5. Empirical performance and validation

OSAQ is evaluated on LLaMA2-7B/13B/70B, LLaMA3-8B/70B, and larger instruction-tuned models including Mistral-Large-123B-Instruct and Llama-3.1-405B-Instruct. The main tasks are WikiText2 and C4 perplexity, zero-shot commonsense QA, MMLU, and MT-Bench. The strongest gains appear in the most aggressive regime, especially $s$4-bit weight-only quantization (Li et al., 6 May 2026).

Setting	Metric	Baseline $s$5 OSAQ
LLaMA2-7B, W2A16 g128	WikiText2 PPL	36.8 $s$6 21.2
LLaMA2-13B, W2A16 g128	WikiText2 PPL	28.1 $s$7 13.4
LLaMA2-70B, W2A16 g128	WikiText2 PPL	19.2 $s$8 10.7
LLaMA2-7B, W2A16 g128 + coordinate descent	WikiText2 PPL	21.2 $s$9 10.6
LLaMA2-13B, W3A16	WikiText2 PPL	6.44 $2$0 5.72
LLaMA3-8B, W3A16 g128	Avg. zero-shot QA	63.6% $2$1 65.2%
LLaMA2-7B, W4A16	MMLU avg. acc.	38.7% $2$2 39.6%

These numbers support the paper’s headline statement that, in 2-bit quantization, OSAQ integrated with GPTQ achieves over 40% lower perplexity compared to vanilla GPTQ. The effect is smaller but still consistent at $2$3 bits and $2$4 bits.

Several analyses are used to justify the mechanism. Weight-distribution visualization shows visibly tightened distributions after the additive transformation. On the original FP16 model, the perturbation changes perplexity only slightly—for example, on LLaMA2-7B WikiText2 it moves from 5.47 to 5.52—while the benefit after quantization is much larger, supporting approximate loss invariance. The null-space stability study measures singular values of $2$5 from different calibration batches and reports maxima around 0.965–0.981 across layers, and around 0.967–0.977 across differing calibration and inference distributions, supporting the stable-null-space hypothesis. A further ablation shows that a direct $2$6 objective is weaker than the proposed Softmax-$2$7+$2$8 approximation: on LLaMA2-7B W3A16, WikiText2 perplexity is 7.82 with direct $2$9 versus 6.11 with the proposed objective (Li et al., 6 May 2026).

The paper also reports compatibility with downstream methods beyond GPTQ. In W4A16KV4, OSAQ+WKVQuant improves over WKVQuant alone; in W4A4, OSAQ improves QuaRot, DuQuant, and SpinQuant modestly but consistently (Li et al., 6 May 2026).

6. Position within outlier-aware quantization research

OSAQ occupies a specific place in the broader outlier-aware quantization landscape. Its defining property is additive, second-order, null-space-guided weight transformation. This differs from multiplicative redistribution methods such as DuQuant, which uses rotation and zigzag permutation to distribute both massive and normal outliers while preserving layer equivalence (Lin et al., 2024). It also differs from explicit preservation and routing approaches such as MicroScopiQ, which retains outliers at higher precision and uses selective pruning plus hardware co-design (Ramachandran et al., 2024), and QMC, which isolates top-$3$0 high-magnitude weights and stores them in MRAM while mapping inliers to ReRAM (Pandey et al., 21 Jan 2026).

The contrast is similarly sharp with decomposition or separation methods. MUXQ detects outlier channels in input activations and rewrites inference as a Body path plus an Aux correction path,

$3$1

which is structurally an auxiliary-branch method rather than self-absorption (Lee et al., 6 Apr 2026). OSC also adopts explicit outlier separation, using a 4-bit GEMM main path plus a 16-bit branch GEMM on a compact tensor of extracted channels (Zhang et al., 14 Apr 2026).

At the same time, some later methods adopt mechanisms closer in spirit to OSAQ’s name than to its exact mathematics. CodeQuant for MoE models states that it absorbs weight outliers into fine-tuned cluster centroids, combining activation smoothing by learnable rotation with centroid-based weight representation (Yin et al., 12 Apr 2026). MorphoQuant, in omni-modal PTQ, selectively absorbs long-tailed outliers into channel-wise biases through Distribution-Aware Bias Compensation (Wu et al., 3 Jun 2026). These works suggest that “absorption” has become a broader design theme, but OSAQ remains distinguished by its stable Hessian null space and closed-form additive solution.

A common misconception is to equate OSAQ with generic outlier smoothing. The paper’s own framing is narrower: OSAQ is not a new quantizer, not a mixed-precision retention scheme, and not an inter-layer equivalent transformation. It is an additive preprocessing step that exploits low-curvature directions of the layer loss landscape.

7. Assumptions, limits, and open scope

OSAQ’s theoretical justification is explicitly approximate. It depends on a second-order view of loss sensitivity, on the Hessian being sufficiently low-rank, and on the low-curvature subspace remaining stable across data. The paper also notes that the “null space” is not exact in a strict linear-algebraic sense; it is formed from small-eigenvalue directions chosen by a tail-energy heuristic. If the Hessian were not sufficiently low-rank, or if its low-curvature subspace changed substantially across data, the safety of the additive perturbation would weaken (Li et al., 6 May 2026).

The empirical evidence mainly covers LLM weight-only PTQ and related quantized settings. The paper does not claim a general theorem for all architectures or all training distributions. Its strongest gains appear in the regimes where existing PTQ methods struggle most, especially $3$2-bit weight-only quantization. This suggests that OSAQ is primarily a remedy for extreme low-bit failure modes rather than a universal replacement for established $3$3-bit or $3$4-bit pipelines.

Within that scope, OSAQ establishes a distinct encyclopedia-worthy formulation: low-bit quantization can be improved by identifying a stable Hessian null space, constructing a closed-form additive transformation inside that space, and using that transformation to suppress weight outliers before conventional PTQ. In the literature of outlier-aware quantization, its importance lies in showing that outlier mitigation need not be multiplicative, need not require a side path, and need not alter inference-time execution at all (Li et al., 6 May 2026).