LoaQ: Layer-wise Output Approximation Quantization

• LoaQ is a post-training quantization method that redefines layer-wise PTQ by aligning quantized layer outputs with the original model outputs for better consistency.
• It employs a closed-form correction technique using activation mismatches, residual adjustments, and tunable interpolation coefficients to refine weight quantization.
• Empirical results show that LoaQ enhances accuracy and perplexity in low-bit quantization for models like LLaMA and Qwen, outperforming traditional methods.

LoaQ, short for Layer-wise Output Approximation Quantization, is a post-training quantization method for LLMs that reformulates layer-wise PTQ around output-level consistency rather than strictly local weight approximation. In place of objectives that only match a layer’s quantized output to its own locally perturbed floating-point computation, LoaQ explicitly targets consistency with the corresponding output of the original model, including residual-bearing sublayer outputs. This produces a closed-form corrected target weight that can be passed to existing activation-aware quantizers, making the method orthogonal to established PTQ backends while remaining within a layer-wise calibration pipeline (Lin et al., 8 Sep 2025).

1. Definition and problem setting

LoaQ is situated in the standard layer-wise PTQ regime for LLMs. In that regime, a pretrained floating-point transformer is quantized one layer or module at a time using calibration data. A typical local objective for a linear transformation $XW$ is to choose a quantized weight $Q$ by minimizing

$\|\hat X Q - \hat X W\|_F^2,$

where $\hat X$ is the layer input produced by the already quantized upstream network. This is activation-aware, but it remains strictly local.

The core critique behind LoaQ is that such local fitting does not directly preserve the behavior of the original full model. Once upstream quantization perturbs the input from $X$ to $\hat X$, matching $\hat XQ$ to $\hat XW$ may preserve the wrong target. LoaQ therefore replaces the local target with the original floating-point output. For input-side linear modules, the intended target is

$\hat X Q \approx XW,$

and for output-side modules with residual paths, the target becomes

$\hat h + \hat X Q \approx h + XW.$

This output-approximation view is motivated by the structural characteristics of mainstream transformer LLMs: stacked transformer blocks, input-side and output-side linear projections, and residual connections whose composed outputs determine downstream behavior. LoaQ keeps the efficiency of layer-wise PTQ, but changes what each local step is trying to preserve (Lin et al., 8 Sep 2025).

2. Mathematical formulation

LoaQ begins from a general least-squares identity. For matrices $Q$0 and $Q$1,

$Q$2

where

$Q$3

This allows output-matching objectives to be converted into activation-aware quantization around a corrected target weight.

For input-side modules, LoaQ uses

$Q$4

With

$Q$5

the objective can be rewritten as

$Q$6

The effective center is therefore

$Q$7

For output-side modules with residuals, LoaQ uses

$Q$8

This becomes

$Q$9

The corrected target now contains both an activation-mismatch term and a residual-mismatch term.

The algorithm uses tunable interpolation coefficients $\|\hat X Q - \hat X W\|_F^2,$0, yielding the practical corrected target

$\|\hat X Q - \hat X W\|_F^2,$1

For input-side modules, $\|\hat X Q - \hat X W\|_F^2,$2. For output-side modules, LoaQ computes token-wise rescaled activations and residuals,

$\|\hat X Q - \hat X W\|_F^2,$3

$\|\hat X Q - \hat X W\|_F^2,$4

$\|\hat X Q - \hat X W\|_F^2,$5

The formulation remains close to GPTQ-style second-order PTQ. Standard activation-aware fitting minimizes

$\|\hat X Q - \hat X W\|_F^2,$6

with $\|\hat X Q - \hat X W\|_F^2,$7. LoaQ preserves this activation-aware geometry but replaces the local center $\|\hat X Q - \hat X W\|_F^2,$8 with the output-consistent target $\|\hat X Q - \hat X W\|_F^2,$9 (Lin et al., 8 Sep 2025).

3. Algorithm and integration into PTQ pipelines

The LoaQ procedure takes as input layer weights

$\hat X$0

calibration inputs $\hat X$1, a bit width $\hat X$2, and the adjustment parameters $\hat X$3.

The algorithm first collects baseline outputs $\hat X$4, where $\hat X$5 denotes outputs from the original floating-point model and $\hat X$6 denotes outputs from the progressively quantized model. It then iterates over layers. For each $\hat X$7, it collects activation pairs $\hat X$8.

For input-side weights $\hat X$9, LoaQ sets

$X$0

forms

$X$1

and then calls an external quantizer: $X$2

For output-side weights $X$3, LoaQ first computes token-wise rescaling,

$X$4

then

$X$5

$X$6

$X$7

and finally

$X$8

After quantizing both submodules of layer $X$9, the algorithm runs forward again to update $\hat X$0 before proceeding to the next layer.

This design makes LoaQ a pipeline modification, not a replacement for existing PTQ solvers. The quantization backend is abstracted as QuantMethod, and the accompanying discussion explicitly connects LoaQ to GPTQ-style methods and to techniques such as Hadamard transforms and NeUQI. The only new ingredients are the corrected target $\hat X$1 and the extra bookkeeping for $\hat X$2. Hyperparameter search is specified as

$\hat X$3

This suggests that the exact closed-form correction is often tempered in practice by interpolation (Lin et al., 8 Sep 2025).

4. Empirical behavior on LLaMA and Qwen models

The reported experiments evaluate 3-bit channel-wise quantization on LLaMA 2, LLaMA 3, and Qwen 3 model families, using Wiki2 and C4 perplexity together with downstream accuracies on ArcC, ArcE, HellaS, PiQA, and WinoG, summarized as Acc.

The strongest gains appear in several low-bit settings where strictly local PTQ is evidently fragile. For LLaMA 2 7B, LoaQ reaches Acc 60.58, compared with 56.89 for GPTQ, 58.74 for GPTAQ, and 57.84 for Qronos; it also attains the best perplexities, Wiki2 7.46 and C4 8.92. For LLaMA 3 8B, LoaQ reaches Acc 57.26, compared with 49.36 for GPTQ, 55.16 for GPTAQ, and 53.15 for Qronos, with markedly better perplexity, Wiki2 12.53 and C4 15.01. For Qwen 3 8B, LoaQ reaches Acc 60.90, exceeding 52.26 for GPTQ, 57.39 for GPTAQ, and 57.53 for Qronos. For Qwen 3 32B, LoaQ reports Acc 64.99, again the best result in the table (Lin et al., 8 Sep 2025).

The gains are more modest on LLaMA 2 13B, LLaMA 2 70B, and Qwen 3 14B. For LLaMA 2 13B, LoaQ gives Acc 63.60, close to 63.63 for GPTAQ, while still giving the best perplexities, Wiki2 6.26 and C4 7.76. For LLaMA 2 70B, all methods are tightly clustered in average accuracy and LoaQ is roughly on par with the best baseline. A particularly difficult regime appears for LLaMA 3 70B at 3-bit channel-wise quantization, where all methods exhibit severe perplexity degradation; LoaQ still slightly improves average task accuracy to 35.54, but this remains a failure-prone operating point for the entire comparison set.

The overall empirical pattern is therefore selective rather than uniform. LoaQ is especially effective in settings where accumulated mismatch from local PTQ appears to be a dominant error source, and more modest where conventional layer-wise solvers are already comparatively stable (Lin et al., 8 Sep 2025).

5. Relation to neighboring quantization methods

LoaQ belongs to the family of activation-aware layer-wise PTQ methods, but its defining distinction is not a new quantizer or codebook. Its contribution is a new target for layer-wise optimization. Standard local PTQ minimizes expressions of the form

$\hat X$4

which preserve the current layer relative to the already distorted input $\hat X$5. LoaQ instead targets the original output, leading to corrected centers such as

$\hat X$6

and

$\hat X$7

This makes it best understood as an output-approximation layer that can be wrapped around GPTQ-like backends rather than as a standalone replacement for them.

A later unification paper, LPCD, places LoaQ in a broader historical sequence from layer-wise PTQ toward submodule-level quantization. In that account, LoaQ is described as targeting Transformer submodules that include a self-attention block, an MLP block, a residual connection, and RMSNorm, with an objective

$\hat X$8

LPCD further rewrites LoaQ as a corrected-target activation-aware problem,

$\hat X$9

with

$\hat XQ$0

In that interpretation, LoaQ remains fundamentally layer-wise, even though it is more output-aware than classical PTQ. LPCD presents itself as a strict generalization: LoaQ becomes a single-step instance of a broader block-coordinate submodule optimization framework rather than a full solution to arbitrary submodule quantization (Ichikawa et al., 1 Dec 2025).

6. Significance, limitations, and interpretation

LoaQ’s significance lies in showing that better local objectives can materially improve PTQ without abandoning layer-wise efficiency. The method does not require end-to-end retraining, and it does not alter the basic computational form of GPTQ-style quantization. Instead, it changes the local fitting target so that each step more faithfully approximates the original model output. This suggests that part of the low-bit PTQ frontier is governed not only by better quantizers, but also by more faithful definitions of what each layer should preserve.

Several limitations are explicit in the reported material. LoaQ is still a layer-wise method; it does not jointly optimize larger coupled structures across the network. It depends on calibration-time quantities $\hat XQ$1, so its effectiveness is tied to the calibration distribution. It also introduces additional bookkeeping relative to plain GPTQ, because both floating-point and quantized activation streams, together with residual outputs, must be tracked during quantization. Finally, the difficult LLaMA 3 70B, 3-bit channel-wise setting indicates that output-level correction does not remove all instability in extreme compression regimes.

A plausible implication is that LoaQ occupies an intermediate point between classical layer-wise PTQ and later submodule-level frameworks: it preserves the simplicity and compatibility of layer-wise solvers, while moving the objective closer to model-level behavior. That intermediate role is exactly how it is later positioned by LPCD, which credits LoaQ for aligning PTQ more closely with residual-path and normalized outputs while also arguing that larger coherent submodules require a more general optimization framework (Ichikawa et al., 1 Dec 2025).