Sequential Adaptive Residual Quantization (SARQ)

• SARQ is a sequential residual quantization method that adaptively determines the number of quantization layers based on path entropy.
• It integrates residual coding with an adaptive stopping criterion by comparing cumulative self-information to budget thresholds, balancing precision and generalization.
• This framework enhances various applications, including retrieval, token-adaptive LLMs, and autoregressive models, by offering dynamic code lengths for head and tail items.

Sequential Adaptive Residual Quantization (SARQ) is a residual quantization scheme in which quantization proceeds level by level and the use of deeper residual levels is determined adaptively rather than fixed a priori. The term appears explicitly in "Towards Efficient and Generalizable Retrieval: Adaptive Semantic Quantization and Residual Knowledge Transfer" (Wang et al., 27 Feb 2026), where SARQ dynamically allocate code lengths based on item path entropy, assigning longer, discriminative IDs to head items and shorter, generalizable IDs to tail items. Closely related mechanisms also appear in "MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs" (Wang et al., 21 Feb 2026), which is not framed using the term “Sequential Adaptive Residual Quantization (SARQ)” but conceptually is very close for LLM weights; in "Autoregressive Image Generation using Residual Quantization" (Lee et al., 2022), which implements sequential residual coding in a two-stage autoregressive image model; and in "Sequential Regression for Continuous Value Prediction using Residual Quantization" (Cui et al., 26 Feb 2026), which proposes a residual quantization–based sequential regression framework that is almost exactly what one might call a Sequential Residual Quantization method and can be turned into a SARQ method with adaptive depth.

1. Definition and conceptual scope

In SA$^2$CRQ, SARQ is the adaptive coding mechanism inside a larger framework built on top of Residual Quantized VAE (RQ-VAE). Corpus items are encoded into continuous embeddings $\mathbf{z}$ and then residually quantized into discrete semantic IDs. SARQ treats quantization as a sequential decision process over levels $l=1,\dots,L$, computes the path entropy of the code prefix, and decides whether to continue or stop. Head items typically reach full depth, whereas tail items terminate early, so the semantic ID length becomes item-dependent rather than globally fixed (Wang et al., 27 Feb 2026).

This definition is narrower than the broader pattern suggested by the surrounding literature. In MoBiQuant, the sequential element is a recursive residual representation of weights, and the adaptive element is a token-aware router that dynamically selects the number of residual bit slices. In RQ-VAE and RQ-Transformer, the sequential element is multi-stage residual coding followed by autoregressive prediction over residual depths. In RQ-Reg, the sequential element is autoregressive prediction of ordered quantization codes, while adaptivity is presented as a natural extension through dynamic stopping (Wang et al., 21 Feb 2026).

A plausible implication is that SARQ is best understood not as a single algorithm but as a family of designs with three invariant components: residual decomposition, ordered refinement, and input- or context-dependent allocation of refinement depth. That interpretation is explicit in retrieval, close to explicit in elastic LLM quantization, and partly inferential in residual-code image generation and sequential regression.

2. Formal mechanism in SA$^2$CRQ

SARQ in SA$^2$CRQ follows standard residual quantization at the level of latent embeddings. Let $\mathbf{z} \in \mathbb{R}^d$ be a continuous latent representation of an item. Residuals are defined recursively by

$$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$

where $\mathbf{q}_l$ is the codeword chosen at layer $l$. The reconstructed latent at depth $k$ is

$\mathbf{z}$0

and the reconstruction term for dynamic depth $\mathbf{z}$1 is

$\mathbf{z}$2

The residual aspect of SARQ is therefore identical to RQ: each layer quantizes the current residual of the previous approximation (Wang et al., 27 Feb 2026).

The sequential and adaptive aspects are defined through item path entropy. Let $\mathbf{z}$3 denote the discrete index chosen at layer $\mathbf{z}$4. For a prefix $\mathbf{z}$5, the self-information is

$\mathbf{z}$6

At each layer $\mathbf{z}$7, SARQ compares this quantity with a layer-specific information budget $\mathbf{z}$8. If

$\mathbf{z}$9

quantization proceeds to layer $l=1,\dots,L$0; otherwise it terminates at depth $l=1,\dots,L$1. Equivalently,

$l=1,\dots,L$2

The paper further reports SARQ configurations such as $l=1,\dots,L$3, $l=1,\dots,L$4, and $l=1,\dots,L$5, and states that the budget $l=1,\dots,L$6 regulates the generalization-precision trade-off, where stricter budgets encourage shorter codes to maximize tail coverage (Wang et al., 27 Feb 2026).

The intended allocation behavior is asymmetric across the long tail. Head items, described as residing in dense regions, consistently satisfy the budget and receive full-depth codes for maximal discriminability. Tail items exceed the budget and trigger early termination, constraining them to generic manifolds to enhance generalization. In this formulation, collision control and generalization are both consequences of the same entropy-thresholded residual coding process rather than separate heuristics (Wang et al., 27 Feb 2026).

3. Codebooks, curriculum, and optimization

SA$l=1,\dots,L$7CRQ couples SARQ with Anchored Curriculum Residual Quantization (ACRQ). During head training, the framework learns hierarchical codebooks $l=1,\dots,L$8 with monotonically increasing capacity,

$l=1,\dots,L$9

The stated interpretation is that small $^2$0 at layer 1 forces a coarse, low-entropy macro semantic space, while larger $^2$1 in deeper layers allow encoding fine-grained residuals. In reported ablations, codebook-size splits include $^2$2, $^2$3, $^2$4, and $^2$5 (Wang et al., 27 Feb 2026).

During tail training, each layer uses a hybrid codebook

$^2$6

where $^2$7 contains frozen head anchors and $^2$8 contains trainable extension codewords. The tail-stage codebook loss applies gradient masking,

$^2$9

so only extension codewords are updated. The depth decision still comes from SARQ, but the probabilities used for path entropy in Stage 2 are the head priors $^2$0, which the paper describes as preventing representational drift (Wang et al., 27 Feb 2026).

The head-stage loss is

$^2$1

and the tail-stage loss has the same depth-truncated structure with the masked codebook term. A defining property is that all reconstruction, codebook, and commitment terms are summed only over layers $^2$2. Early termination is therefore not merely an inference-time decision; it changes which residual levels participate in representation learning (Wang et al., 27 Feb 2026).

This design makes the adaptive depth controller and the residual codebooks mutually dependent. ACRQ provides frozen geometry and priors; SARQ uses those priors to make entropy-based depth decisions; and the effective supervision at each item is truncated by the depth decision. The result is a joint mechanism for allocating semantic-ID length while regularizing the tail with a head-derived manifold (Wang et al., 27 Feb 2026).

4. Related formulations across domains

The explicit term SARQ is specific to SA$^2$3CRQ, but several contemporaneous systems implement closely related structures.

Paper	Domain	SARQ-relevant mechanism
"MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Elastic LLMs" (Wang et al., 21 Feb 2026)	Elastic LLM inference	Recursive residual quantization of weights plus token-aware routing of residual bit slices
"Autoregressive Image Generation using Residual Quantization" (Lee et al., 2022)	Autoregressive image generation	Multi-stage residual quantization of latent features plus autoregressive prediction over residual depths
"Sequential Regression for Continuous Value Prediction using Residual Quantization" (Cui et al., 26 Feb 2026)	Continuous value prediction in recommendation	Ordered residual quantization codes predicted autoregressively, with adaptive depth proposed as a natural extension

In MoBiQuant, MoBiSlice decomposes a weight matrix as

$^2$4

with

$^2$5

and a target effective precision is reconstructed by

$^2$6

MoBiRoute maps token embeddings to slice scores through a 2-layer MLP, uses a differentiable gate during training, and at inference uses

$^2$7

The paper explicitly interprets this as a sequential adaptive residual quantization mechanism for LLM weights: a recursive residual representation of weights and a token-adaptive policy that sequentially activates a subset of residual levels (Wang et al., 21 Feb 2026).

In RQ-VAE and RQ-Transformer, residual quantization is defined by

$^2$8

with partial reconstructions

$^2$9

The autoregressive latent distribution is factorized as

$\mathbf{z} \in \mathbb{R}^d$0

so the model predicts the residual code stack one depth at a time at each spatial location. The paper’s own synthesis states that this directly maps to what one could call Sequential Adaptive Residual Quantization, with the caveat that the adaptivity is mostly implicit in depth specialization rather than explicit early stopping (Lee et al., 2022).

In RQ-Reg, a target scalar is decomposed by residual quantization into

$\mathbf{z} \in \mathbb{R}^d$1

and the model learns

$\mathbf{z} \in \mathbb{R}^d$2

The framework uses a 2-layer MLP encoder, a single-layer LSTM, a 2-layer MLP predictor, and a 2-layer MLP regressor, with $\mathbf{z} \in \mathbb{R}^d$3 and $\mathbf{z} \in \mathbb{R}^d$4 in experiments. The paper states that the current system is hierarchical but not explicitly adaptive in depth, and then lists dynamic stopping criterion, confidence-based truncation, and instance-dependent $\mathbf{z} \in \mathbb{R}^d$5 as straightforward SARQ extensions (Cui et al., 26 Feb 2026).

5. Empirical behavior and trade-offs

The clearest direct evidence for SARQ comes from SA$\mathbf{z} \in \mathbb{R}^d$6CRQ’s industrial retrieval results.

Variant	Hallu	R@2k	Ret-Per
TIGER	0.2514	0.6448	509
SARQ only ($\mathbf{z} \in \mathbb{R}^d$7)	0.1797	0.6388	555
SARQ only ($\mathbf{z} \in \mathbb{R}^d$8)	0.1739	0.6453	579
SARQ only ($\mathbf{z} \in \mathbb{R}^d$9)	0.1556	0.6798	661
ACRQ only	0.2225	0.6662	612
Full SA$$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$0CRQ ($$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$1)	0.1808	0.7231	914

On the JD.com industrial dataset, the best reported SA$$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$2CRQ configuration improves overall R@2k from $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$3 to $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$4 and Ret-Per from $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$5 to $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$6. The same configuration raises head R@2k from $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$7 to $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$8 and tail R@2k from $$\mathbf{r}_0 = \mathbf{z}, \qquad \mathbf{r}_l = \mathbf{r}_{l-1} - \mathbf{q}_l,$$9 to $\mathbf{q}_l$0. The paper further states that pure SARQ improves tail coverage substantially while keeping head performance competitive, and that increasing $\mathbf{q}_l$1 increases Ret-Per and R@2k, consistent with deeper codes giving more fine-grained indexing (Wang et al., 27 Feb 2026).

Outside retrieval, the closest SARQ-like evidence comes from MoBiQuant. The framework uses a default configuration with $\mathbf{q}_l$2 slices and $\mathbf{q}_l$3 bits per slice, so 1, 2, 3, or 4 slices yield 2, 4, 6, or 8 effective bits. It reports that MoBiQuant exhibits strong elasticity, enabling it to match the performance of bit-specific calibrated PTQ on LLaMA3-8B without repeated calibration, and that memory and time cost scale roughly with the average number of active slices. The paper reports up to $\mathbf{q}_l$4 speed-up over FP16 on A100 for LLaMA-2-7B. It also reports, on LLaMA3-8B WikiText2 perplexity, FP16 at $\mathbf{q}_l$5, OmniQuant $\mathbf{q}_l$6b at $\mathbf{q}_l$7, OmniQuant $\mathbf{q}_l$8b at $\mathbf{q}_l$9, and MoBiQuant at average $l$0 bits with $l$1 PPL and average $l$2 bits with $l$3 PPL (Wang et al., 21 Feb 2026).

The image-generation and sequential-regression papers support a broader interpretation of the same design pattern. RQ-VAE shows that increasing residual depth $l$4 is a more effective way to improve rate–distortion than growing the codebook size $l$5, and RQ-Transformer reports $l$6–$l$7 faster sampling than VQ-GAN at 1.4B parameters. RQ-Reg reports that RQ K-means with $l$8 yields reconstruction error $l$9 MAE on KuaiRec versus $k$0 for K-means, and that more RQ layers improve prediction performance (Lee et al., 2022, Cui et al., 26 Feb 2026).

6. Limitations, misconceptions, and open directions

A common misconception is to treat SARQ as synonymous with any residual quantization scheme. The current literature suggests a narrower usage. Residual quantization by itself is not sufficient: SARQ additionally requires that residual levels be used sequentially and that continuation to deeper levels be determined adaptively. In SA$k$1CRQ this adaptivity is explicit through path entropy; in MoBiQuant it is explicit through token-aware routing; in RQ-VAE and RQ-Reg it is largely interpretive or prospective rather than part of the baseline algorithm (Wang et al., 27 Feb 2026).

The main practical limitations are domain-specific. For SA$k$2CRQ, the text implies hyperparameter sensitivity in the global budget parameter $k$3 and in codebook splits $k$4, as well as reliance on accurate path probabilities learned from head items. For MoBiQuant, the stated limitations include router and calibration overhead, complex kernel requirements, routing granularity that is not strictly hierarchical, and target bit sensitivity. For RQ-Reg, the limitations include a fixed number of levels $k$5, global codebooks built offline, and the absence of an end-to-end differentiable quantizer with adaptive depth. For RQ-Transformer, the paper notes that autoregressive models can overfit on small datasets, that AR context is unidirectional, and that training large AR plus SARQ-like models is still computationally and environmentally expensive (Wang et al., 21 Feb 2026, Lee et al., 2022, Cui et al., 26 Feb 2026).

Several future directions recur across the papers. SA$k$6CRQ explicitly identifies extension to multi-modal and cross-domain retrieval scenarios. MoBiQuant suggests hierarchical gating, more explicit error-based routing loss, and joint weight+activation residual quantization. RQ-Reg lists dynamic stopping criterion, confidence-based truncation, instance-dependent depth, task-dependent or instance-dependent codebooks, and enhanced ordinal or tail-aware objectives. RQ-Transformer suggests varying depth $k$7 adaptively per position based on residual magnitude. This suggests that the central open problem is not whether residual quantization can be made sequential, but how to make the stopping rule, budget allocation, and residual code geometry jointly learnable without losing the efficiency and regularization benefits that motivate SARQ in the first place (Wang et al., 27 Feb 2026, Wang et al., 21 Feb 2026, Cui et al., 26 Feb 2026, Lee et al., 2022).