---
title: 'LoaQ: Layer-wise Output Approximation Quantization'
url: https://www.emergentmind.com/topics/loaq
type: topic
---

# LoaQ: Layer-wise Output Approximation Quantization

LoaQ, short for **Layer-wise Output Approximation Quantization**, is a post-training quantization method for large language models that reformulates layer-wise PTQ around **output-level consistency** rather than strictly local weight approximation. In place of objectives that only match a layer’s quantized output to its own locally perturbed floating-point computation, LoaQ explicitly targets consistency with the corresponding output of the original model, including residual-bearing sublayer outputs. This produces a closed-form corrected target weight that can be passed to existing activation-aware quantizers, making the method orthogonal to established PTQ backends while remaining within a layer-wise calibration pipeline [2509.06297].

## 1. Definition and problem setting

LoaQ is situated in the standard **layer-wise PTQ** regime for LLMs. In that regime, a pretrained floating-point transformer is quantized one layer or module at a time using calibration data. A typical local objective for a linear transformation \(XW\) is to choose a quantized weight \(Q\) by minimizing
\[
\|\hat X Q - \hat X W\|_F^2,
\]
where \(\hat X\) is the layer input produced by the already quantized upstream network. This is activation-aware, but it remains strictly local.

The core critique behind LoaQ is that such local fitting does not directly preserve the behavior of the original full model. Once upstream quantization perturbs the input from \(X\) to \(\hat X\), matching \(\hat XQ\) to \(\hat XW\) may preserve the wrong target. LoaQ therefore replaces the local target with the original floating-point output. For input-side linear modules, the intended target is
\[
\hat X Q \approx XW,
\]
and for output-side modules with residual paths, the target becomes
\[
\hat h + \hat X Q \approx h + XW.
\]

This output-approximation view is motivated by the structural characteristics of mainstream transformer LLMs: stacked transformer blocks, input-side and output-side linear projections, and residual connections whose composed outputs determine downstream behavior. LoaQ keeps the efficiency of layer-wise PTQ, but changes what each local step is trying to preserve [2509.06297].

## 2. Mathematical formulation

LoaQ begins from a general least-squares identity. For matrices \(X\) and \(Y\),
\[
\arg\min_{Q} \|XQ - Y\|_{F}^{2}
=
\arg\min_{Q} \Bigl\| X\bigl(Q - Q^{*}\bigr) \Bigr\|_{F}^{2},
\]
where
\[
Q^{*} = (X^{\top}X)^{-1}X^{\top}Y.
\]

This allows output-matching objectives to be converted into activation-aware quantization around a corrected target weight.

For **input-side modules**, LoaQ uses
\[
\arg\min_Q \|\hat XQ- XW\|_F^2.
\]
With
\[
H = \hat X^\top \hat X,\qquad C = \hat X^\top (X-\hat X),
\]
the objective can be rewritten as
\[
\arg\min\|\hat XQ- XW\|_F^2
=
\arg\min\|\hat X\big(Q-(I+H^{-1}C)W\big)\|_F^2.
\]
The effective center is therefore
\[
\widetilde W = (I + H^{-1}C)W.
\]

For **output-side modules with residuals**, LoaQ uses
\[
\arg\min\|(\hat h + \hat XQ)- (h + XW)\|_F^2.
\]
This becomes
\[
\arg\min\|\hat X\big(Q-((I+H^{-1}C)W + H^{-1}\hat X^T(h-\hat h))\big)\|_F^2.
\]
The corrected target now contains both an activation-mismatch term and a residual-mismatch term.

The algorithm uses tunable interpolation coefficients \(\alpha,\beta\in[0,1]\), yielding the practical corrected target
\[
\widetilde W \leftarrow \bigl(I+\alpha H^{-1}C\bigr)W+\beta\Delta.
\]
For input-side modules, \(\Delta=0\). For output-side modules, LoaQ computes token-wise rescaled activations and residuals,
\[
X' \leftarrow sX,\qquad \hat X' \leftarrow \hat s\hat X,
\]
\[
H\leftarrow \hat X'^\top \hat X',\qquad
C\leftarrow \hat X'^\top(X'-\hat X'),
\]
\[
\Delta\leftarrow H^{-1}\hat X'^\top\bigl(s h-\hat s\hat h\bigr).
\]

The formulation remains close to GPTQ-style second-order PTQ. Standard activation-aware fitting minimizes
\[
\|\hat X(Q-W)\|_F^2
=
\operatorname{tr}\big((Q-W)^\top H (Q-W)\big),
\]
with \(H=\hat X^\top\hat X\). LoaQ preserves this activation-aware geometry but replaces the local center \(W\) with the output-consistent target \(\widetilde W\) [2509.06297].

## 3. Algorithm and integration into PTQ pipelines

The LoaQ procedure takes as input layer weights
\[
\{W_\mathrm{in}^{(\ell)},W_\mathrm{out}^{(\ell)}\}_{\ell=1}^L,
\]
calibration inputs \(\{I^{(i)}\}_{i=1}^S\), a bit width \(k\), and the adjustment parameters \(\alpha,\beta\).

The algorithm first collects baseline outputs \(h,\hat h\), where \(h\) denotes outputs from the original floating-point model and \(\hat h\) denotes outputs from the progressively quantized model. It then iterates over layers. For each \(W \in \{W_\mathrm{in}^{(\ell)},W_\mathrm{out}^{(\ell)}\}\), it collects activation pairs \(X,\hat X\).

For **input-side weights** \(W_\mathrm{in}^{(\ell)}\), LoaQ sets
\[
H \leftarrow \hat X^\top \hat X,\qquad C \leftarrow \hat X^\top (X-\hat X),\qquad \Delta = 0,
\]
forms
\[
\widetilde W \leftarrow \bigl(I+\alpha H^{-1}C\bigr)W,
\]
and then calls an external quantizer:
\[
Q \leftarrow \mathrm{QuantMethod}(\widetilde W;H;k).
\]

For **output-side weights** \(W_\mathrm{out}^{(\ell)}\), LoaQ first computes token-wise rescaling,
\[
s\leftarrow R(h+XW), \qquad \hat s\leftarrow R(h+\hat XW),
\]
then
\[
X' \leftarrow sX, \qquad \hat X' \leftarrow \hat s\hat X,
\]
\[
H\leftarrow\hat X'^\top\hat X', \qquad C\leftarrow\hat X'^\top(X'-\hat X'),
\]
\[
\Delta\leftarrow H^{-1}\hat X'^\top\bigl(s h-\hat s\hat h\bigr),
\]
and finally
\[
\widetilde W\leftarrow\bigl(I+\alpha H^{-1}C\bigr)W+\beta\Delta,
\qquad
Q\leftarrow\mathrm{QuantMethod}(\widetilde W;H;k).
\]

After quantizing both submodules of layer \(\ell\), the algorithm runs forward again to update \(h,\hat h\) before proceeding to the next layer.

This design makes LoaQ a **pipeline modification**, not a replacement for existing PTQ solvers. The quantization backend is abstracted as `QuantMethod`, and the accompanying discussion explicitly connects LoaQ to GPTQ-style methods and to techniques such as Hadamard transforms and NeUQI. The only new ingredients are the corrected target \(\widetilde W\) and the extra bookkeeping for \(X,\hat X,h,\hat h\). Hyperparameter search is specified as
\[
\alpha \in \{\,0.1 \times i \mid i = 0,1,2,\ldots,10\,\},
\qquad
\beta  \in \{\,0.05 \times j \mid j = 0,1,2,\ldots,20\,\}.
\]
This suggests that the exact closed-form correction is often tempered in practice by interpolation [2509.06297].

## 4. Empirical behavior on LLaMA and Qwen models

The reported experiments evaluate **3-bit channel-wise quantization** on LLaMA 2, LLaMA 3, and Qwen 3 model families, using **Wiki2** and **C4** perplexity together with downstream accuracies on **ArcC**, **ArcE**, **HellaS**, **PiQA**, and **WinoG**, summarized as **Acc**.

The strongest gains appear in several low-bit settings where strictly local PTQ is evidently fragile. For **LLaMA 2 7B**, LoaQ reaches **Acc 60.58**, compared with **56.89** for GPTQ, **58.74** for GPTAQ, and **57.84** for Qronos; it also attains the best perplexities, **Wiki2 7.46** and **C4 8.92**. For **LLaMA 3 8B**, LoaQ reaches **Acc 57.26**, compared with **49.36** for GPTQ, **55.16** for GPTAQ, and **53.15** for Qronos, with markedly better perplexity, **Wiki2 12.53** and **C4 15.01**. For **Qwen 3 8B**, LoaQ reaches **Acc 60.90**, exceeding **52.26** for GPTQ, **57.39** for GPTAQ, and **57.53** for Qronos. For **Qwen 3 32B**, LoaQ reports **Acc 64.99**, again the best result in the table [2509.06297].

The gains are more modest on **LLaMA 2 13B**, **LLaMA 2 70B**, and **Qwen 3 14B**. For **LLaMA 2 13B**, LoaQ gives **Acc 63.60**, close to **63.63** for GPTAQ, while still giving the best perplexities, **Wiki2 6.26** and **C4 7.76**. For **LLaMA 2 70B**, all methods are tightly clustered in average accuracy and LoaQ is roughly on par with the best baseline. A particularly difficult regime appears for **LLaMA 3 70B at 3-bit channel-wise quantization**, where all methods exhibit severe perplexity degradation; LoaQ still slightly improves average task accuracy to **35.54**, but this remains a failure-prone operating point for the entire comparison set.

The overall empirical pattern is therefore selective rather than uniform. LoaQ is especially effective in settings where accumulated mismatch from local PTQ appears to be a dominant error source, and more modest where conventional layer-wise solvers are already comparatively stable [2509.06297].

## 5. Relation to neighboring quantization methods

LoaQ belongs to the family of **activation-aware layer-wise PTQ** methods, but its defining distinction is not a new quantizer or codebook. Its contribution is a new **target** for layer-wise optimization. Standard local PTQ minimizes expressions of the form
\[
\|\hat X(Q-W)\|_F^2,
\]
which preserve the current layer relative to the already distorted input \(\hat X\). LoaQ instead targets the original output, leading to corrected centers such as
\[
(I+H^{-1}C)W
\]
and
\[
(I+H^{-1}C)W + H^{-1}\hat X^\top(h-\hat h).
\]
This makes it best understood as an output-approximation layer that can be wrapped around GPTQ-like backends rather than as a standalone replacement for them.

A later unification paper, **LPCD**, places LoaQ in a broader historical sequence from layer-wise PTQ toward submodule-level quantization. In that account, LoaQ is described as targeting Transformer submodules that include a self-attention block, an MLP block, a residual connection, and RMSNorm, with an objective
\[
\widehat{W}^{\mathrm{LoaQ}}
=
\arg\min_{\widehat{W}}
\|(\widehat R+\widehat X\widehat W)-(R+XW)\|_F^2.
\]
LPCD further rewrites LoaQ as a corrected-target activation-aware problem,
\[
\widehat{W}^{\mathrm{LoaQ}}
=
\arg\min_{\widehat{W}}
\|\widehat X(\widehat W-W^\ast(\alpha,\beta))\|_F^2,
\]
with
\[
W^\ast(\alpha,\beta) = (I+\alpha \widehat H^{-1}C)W+\beta \widehat H^{-1}\Gamma,
\qquad
\Gamma=\widehat X^\top(R-\widehat R).
\]
In that interpretation, LoaQ remains fundamentally **layer-wise**, even though it is more output-aware than classical PTQ. LPCD presents itself as a strict generalization: LoaQ becomes a single-step instance of a broader block-coordinate submodule optimization framework rather than a full solution to arbitrary submodule quantization [2512.01546].

## 6. Significance, limitations, and interpretation

LoaQ’s significance lies in showing that **better local objectives** can materially improve PTQ without abandoning layer-wise efficiency. The method does not require end-to-end retraining, and it does not alter the basic computational form of GPTQ-style quantization. Instead, it changes the local fitting target so that each step more faithfully approximates the original model output. This suggests that part of the low-bit PTQ frontier is governed not only by better quantizers, but also by more faithful definitions of what each layer should preserve.

Several limitations are explicit in the reported material. LoaQ is still a **layer-wise** method; it does not jointly optimize larger coupled structures across the network. It depends on calibration-time quantities \(X,\hat X,h,\hat h\), so its effectiveness is tied to the calibration distribution. It also introduces additional bookkeeping relative to plain GPTQ, because both floating-point and quantized activation streams, together with residual outputs, must be tracked during quantization. Finally, the difficult **LLaMA 3 70B, 3-bit channel-wise** setting indicates that output-level correction does not remove all instability in extreme compression regimes.

A plausible implication is that LoaQ occupies an intermediate point between classical layer-wise PTQ and later submodule-level frameworks: it preserves the simplicity and compatibility of layer-wise solvers, while moving the objective closer to model-level behavior. That intermediate role is exactly how it is later positioned by LPCD, which credits LoaQ for aligning PTQ more closely with residual-path and normalized outputs while also arguing that larger coherent submodules require a more general optimization framework [2512.01546].

Source: https://www.emergentmind.com/topics/loaq