ReVQ: Quantize-then-Rectify for Efficient VQ-VAE

• The paper introduces ReVQ, which converts continuous VAE latents into discrete tokens and learns a lightweight post rectifier to correct quantization error, drastically reducing compute.
• ReVQ is a design pattern where a discrete bottleneck is followed by a corrective stage, applicable across domains such as image compression, neural audio codecs, and LLM quantization.
• Experiments demonstrate competitive reconstruction quality (e.g., rFID = 1.06 on ImageNet) while significantly cutting training time compared to standard VQ-VAE methods.

Searching arXiv for the cited papers to ground the article. {"^^^^^^^^5query5 OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^^^^^"," OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^} {"^^^^^^^^5query5 Efficient VQ-VAE Training^^^^^^^^5\5 Quantize-then-Rectify (ReVQ) denotes a class of two-stage quantization procedures in which a continuous object is first mapped to a discrete representation and is then corrected by a secondary mechanism that targets the dominant quantization-induced error. In the formulation introduced for efficient VQ-VAE training, ReVQ converts a pre-trained continuous VAE into a discrete VQ-VAE by quantizing the frozen VAE latent and learning a lightweight post rectifier that pushes the quantized latent back into the decoder’s acceptable manifold (&&&^^^^5query5^^^^&&&). In the broader literature represented here, closely related quantize-then-rectify designs appear in vector quantization for inner-product preservation, post-training quantization for LLMs, latent-space image compression, residual vector quantization for neural audio codecs, and quantized compressed sensing (&&&^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^&&&, &&&^^^^5max_results5^^^^&&&, Luo et al., 2024, &&&^^^^5\5^^^^&&&, &&&^^^^5query5^^^^&&&).

^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^. Conceptual structure and terminology

Across these works, the defining structure is consistent: the quantization stage enforces a bitrate, token, or numerical precision constraint, while the rectification stage compensates for the specific failure mode introduced by that constraint. The rectifier may be a learned residual predictor, a low-rank correction, a ^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^-bit residual estimator, a beam-search refinement, or a convex decoder based on rectified linear penalties. What is rectified is not uniform across domains: in some cases it is latent reconstruction error, in others inner-product bias, weight perturbation, or suboptimal discrete code selection.

Setting	Quantized object	Rectification mechanism
Efficient VQ-VAE training (&&&^^^^5query5^^^^&&&)	Frozen VAE latents	Lightweight post rectifier PRESERVED_PLACEHOLDER_^^^^5query5^^^^ in latent space
TurboQuant (&&&^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^&&&)	Rotated vector coordinates	^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^-bit QJL correction on the residual
LLM PTQ / QER baseline (&&&^^^^5max_results5^^^^&&&)	Weight matrix PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^	Low-rank correction PRESERVED_PLACEHOLDER_^^^^5max_results5^^^^ after quantization
Neural image compression (Luo et al., 2024)	Latent PRESERVED_PLACEHOLDER_^^^^5query5^^^^	Quantization Rectifier PRESERVED_PLACEHOLDER_^^^^5\5^^^^ before synthesis
RVQ neural codecs (&&&^^^^5\5^^^^&&&)	Level-wise RVQ indices	Beam-search refinement of code combinations
Quantized compressed sensing (&&&^^^^5query5^^^^&&&)	One-bit or multi-bit measurements	Convex ReLU-based reconstruction program

This common structure makes ReVQ less a single algorithm than a design pattern. The quantizer establishes the discrete bottleneck; the rectifier is then tailored to the quantity one wishes to preserve under that bottleneck.

^^^^5max_results5^^^^. ReVQ for efficient VQ-VAE training

In "Quantize-then-Rectify: Efficient VQ-VAE Training" (&&&^^^^5query5^^^^&&&), ReVQ is a fast, tolerance-aware method for turning a pre-trained continuous VAE into a discrete VQ-VAE. The central observation is that modern VAEs such as DC-AE exhibit strong noise tolerance: when Gaussian noise is injected into DC-AE’s ^^^^5max_results5query5\5^^^^8-D latent, reconstructions remain visually high-quality for noise variance $\le 0.3$ and degrade beyond this, while the appendix uses latent MSE $\le 0.1$ as a practical acceptability criterion. This motivates a training strategy that does not retrain the heavy VAE, but instead keeps quantization error within the decoder’s tolerance and learns a small corrector for the residual error.

The pipeline is explicit. A pre-trained DC-AE encoder maps an input image $x$ to a latent $z \in \mathbb{R}^{2048}$, normalized by global dataset mean and variance. The VAE encoder and decoder are frozen. The normalized latent is discretized by channel multi-group quantization, and the quantized latent is then processed by a lightweight post rectifier $g$, implemented as an EfficientViT block with matched input and output dimensionality and no up- or down-sampling. The rectified latent is finally passed through the frozen decoder. Formally, if PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^ and PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^, the training objective is

PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^

and the optimization problem is

PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^

No perceptual, adversarial, or commitment losses are used; nearest-neighbor assignments are handled with standard straight-through behavior, while the VAE itself is treated as a black box (&&&^^^^5query5^^^^&&&).

This formulation shifts the optimization burden from full-resolution image space to low-resolution latent space. A plausible implication is that ReVQ’s efficiency is not merely an implementation detail but follows directly from the decision to freeze the compute-heavy encoder and decoder and to optimize only the latent quantizer and rectifier.

^^^^5query5^^^^. Quantization geometry, code capacity, and stabilization

The ReVQ quantizer in (&&&^^^^5query5^^^^&&&) partitions the latent PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5\5^^^^ into PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)55^^^^ channel groups, with per-group dimensionalities PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^6 such that PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^7. Each group has its own codebook PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^8, and each group slice PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^9 is quantized by nearest-neighbor search: PRESERVED_PLACEHOLDER_^^^^5max_results5query5^^^^ The full quantized representation is the concatenation

PRESERVED_PLACEHOLDER_^^^^5max_results5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^

For DC-AE on ImageNet, PRESERVED_PLACEHOLDER_^^^^5max_results5max_results5^^^^ and PRESERVED_PLACEHOLDER_^^^^5max_results5query5^^^^, so the token count PRESERVED_PLACEHOLDER_^^^^5max_results5\5^^^^ is taken to equal the number of channel groups PRESERVED_PLACEHOLDER_^^^^5max_results55^^^^. Each group therefore corresponds to one token index. The reported configurations are PRESERVED_PLACEHOLDER_^^^^5max_results5^^^^6 and PRESERVED_PLACEHOLDER_^^^^5max_results5^^^^7. With PRESERVED_PLACEHOLDER_^^^^5max_results5^^^^8, each token covers PRESERVED_PLACEHOLDER_^^^^5max_results5^^^^9 channels and each group uses PRESERVED_PLACEHOLDER_^^^^5query5query5^^^^ codes; with PRESERVED_PLACEHOLDER_^^^^5query5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^, each token covers PRESERVED_PLACEHOLDER_^^^^5query5max_results5^^^^ channels and each group uses either PRESERVED_PLACEHOLDER_^^^^5query5query5^^^^ or PRESERVED_PLACEHOLDER_^^^^5query5\5^^^^ codes. If all groups share PRESERVED_PLACEHOLDER_^^^^5query55^^^^ entries, the number of code combinations per position is PRESERVED_PLACEHOLDER_^^^^5query5^^^^6. The paper gives the capacity intuition that for PRESERVED_PLACEHOLDER_^^^^5query5^^^^7 and a single shared codebook size PRESERVED_PLACEHOLDER_^^^^5query5^^^^8, the combinations scale as PRESERVED_PLACEHOLDER_^^^^5query5^^^^9 ImageNet’s ^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^.^^^^5max_results5^^^^8M images, while arguing that conventional single-codebook spatial quantizers underutilize this capacity because of symmetry bias (&&&^^^^5query5^^^^&&&).

Two implementation choices are particularly important. First, channel splitting consistently outperforms spatial splitting: for PRESERVED_PLACEHOLDER_^^^^5\5query5^^^^, rFID is PRESERVED_PLACEHOLDER_^^^^5\5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ with channel splitting versus PRESERVED_PLACEHOLDER_^^^^5\5max_results5^^^^ with spatial splitting, and for PRESERVED_PLACEHOLDER_^^^^5\5query5^^^^, rFID is PRESERVED_PLACEHOLDER_^^^^5\5\5^^^^ versus PRESERVED_PLACEHOLDER_^^^^5\55^^^^. Second, ReVQ uses a Non-Activation Reset at the end of each epoch to prevent index collapse. Without reset, codebook utilization falls to PRESERVED_PLACEHOLDER_^^^^5\5^^^^6 as codebook size grows; with reset, utilization stays above PRESERVED_PLACEHOLDER_^^^^5\5^^^^7. In a ^^^^5max_results5^^^^D synthetic setting, the same mechanism reduces quantization error from PRESERVED_PLACEHOLDER_^^^^5\5^^^^8 to PRESERVED_PLACEHOLDER_^^^^5\5^^^^9. The rectifier also matters materially: for $\le 0.3$^^^^5query5^^^^, a ViT-based rectifier gives rFID $\le 0.3$^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^, compared with $\le 0.3$^^^^5max_results5^^^^ for a CNN and $\le 0.3$^^^^5query5^^^^ for an MLP; for $\le 0.3$^^^^5\5^^^^, the corresponding values are $\le 0.3$5, $\le 0.3$6, and $\le 0.3$7. The paper further reports that adding another encoder before the quantizer is harmful, with rFID jumping to approximately $\le 0.3$8 for $\le 0.3$9, which is why the design remains rectifier-only (&&&^^^^5query5^^^^&&&).

^^^^5\5^^^^. Reconstruction quality and compute profile

ReVQ is reported to compress ImageNet images into at most $\le 0.1$^^^^5query5^^^^ tokens while sustaining competitive reconstruction quality, with ReVQ$\le 0.1$^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ reaching rFID $\le 0.1$^^^^5max_results5^^^^ (&&&^^^^5query5^^^^&&&). The reported ImageNet validation metrics and training times are as follows.

Variant	Token / codebook setting	Reported outcome
ReVQ$\le 0.1$^^^^5query5^^^^	$\le 0.1$^^^^5\5^^^^	SSIM ^^^^5query5^^^^.69^^^^5query5 PSNR ^^^^5max_results5query5^^^^.7^^^^5query5query5 LPIPS ^^^^5query5^^^^.^^^^5query5Improving Test-Time Performance of RVQ-based Neural Codecs5max_results5^^^^, rFID ^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^.^^^^5query5^^^^6, $\le 0.1$5 hours on a single NVIDIA RTX ^^^^5\5query5Improving Test-Time Performance of RVQ-based Neural Codecs5query5^^^^
ReVQ$\le 0.1$6	$\le 0.1$7	SSIM ^^^^5query5^^^^.6^^^^5max_results5query5 PSNR ^^^^5max_results5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^.69^^^^5query5^^^^, LPIPS ^^^^5query5^^^^.^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^9, rFID ^^^^5max_results5^^^^.57, $\le 0.1$8 hours
ReVQ$\le 0.1$9	$x$^^^^5query5^^^^	SSIM ^^^^5query5^^^^.6^^^^5\5query5 PSNR ^^^^5max_results5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^.96^^^^5query5^^^^, LPIPS ^^^^5query5^^^^.^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^, rFID ^^^^5max_results5^^^^.^^^^5query55
ReVQ$x$^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ variant	$x$^^^^5max_results5^^^^	$x$^^^^5query5^^^^ hours

The efficiency claim is central. ReVQ completes full training on a single NVIDIA ^^^^5\5query5Improving Test-Time Performance of RVQ-based Neural Codecs5query5^^^^ in approximately $x$^^^^5\5^^^^ hours, whereas the cited comparison point for MaskBit is $x$5 GPU hours, described as $x$6 days on $x$7A^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5query5^^^^ for $x$8M iterations. The reported explanation is architectural: compute-heavy shallow layers of image-space encoders and decoders dominate FLOPs in standard VQ-VAEs, whereas ReVQ freezes the pre-trained VAE and trains only a low-resolution latent quantizer and rectifier. The omission of GAN and perceptual branches, and the absence of encoder and decoder gradients, further reduce memory and wall-clock time (&&&^^^^5query5^^^^&&&).

Against other tokenizers, the paper reports representative rFID values of $x$9 for ViT-VQGAN with $z \in \mathbb{R}^{2048}$^^^^5query5^^^^ tokens, $z \in \mathbb{R}^{2048}$^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ for Mo-VQGAN with $z \in \mathbb{R}^{2048}$^^^^5max_results5^^^^ tokens, $z \in \mathbb{R}^{2048}$^^^^5query5^^^^ for MaskBit with $z \in \mathbb{R}^{2048}$^^^^5\5^^^^ tokens, $z \in \mathbb{R}^{2048}$5 for TiTok-S-^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^8 with $z \in \mathbb{R}^{2048}$6 tokens, $z \in \mathbb{R}^{2048}$7 for CODA with $z \in \mathbb{R}^{2048}$8 tokens, and $z \in \mathbb{R}^{2048}$9 for TokenBridge with $g$^^^^5query5^^^^ tokens. Within that comparison set, ReVQ$g$^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ is presented as offering short sequences, competitive reconstruction, and single-GPU training (&&&^^^^5query5^^^^&&&).

5. Domain-specific variants of the quantize-then-rectify pattern

The same quantize-then-rectify logic appears in several adjacent areas, but the object being preserved differs from case to case.

TurboQuant: TurboQuant first applies a random orthogonal transform, then performs per-coordinate $g$^^^^5max_results5^^^^-bit Lloyd-Max scalar quantization to minimize MSE, and finally rectifies the systematic inner-product bias of the MSE quantizer by adding an unbiased estimate of the residual inner product obtained from a ^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^-bit Quantized Johnson–Lindenstrauss transform (&&&^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^&&&). The paper states that the MSE distortion satisfies

$g$^^^^5query5^^^^

while any randomized vector quantizer obeys the lower bound

$g$^^^^5\5^^^^

so TurboQuant is within the universal constant factor $g$5 of the lower bound. In KV-cache quantization, it reports absolute quality neutrality at $g$6 bits per channel and marginal quality degradation at $g$7 bits per channel.

LLM post-training quantization and QER: In the SRR framework, ReVQ refers to the quantize-then-rectify baseline in which a full-precision weight matrix is approximated as

$g$8

with the full rank budget devoted to fitting the quantization residual after quantization (&&&^^^^5max_results5^^^^&&&). SRR generalizes this by preserving a top-$g$9 singular subspace before quantization and using the remaining rank PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5query5^^^^ for quantization error reconstruction afterward. In that formulation, ReVQ is exactly the PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^ case. The paper reports consistent perplexity reductions in PTQ and a PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5max_results5^^^^ percentage-point average gain on GLUE under PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5query5^^^^-bit QPEFT.

Neural image compression: "Neural Image Compression with Quantization Rectifier" inserts a latent-space predictor between quantization and synthesis, with

PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5\5^^^^

while leaving the entropy model and bitstream unchanged (Luo et al., 2024). The full training objective is

PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query55^^^^

On Kodak, the paper reports universal rate-distortion gains at unchanged bpp, including for Attn+QR an average PSNR gain of approximately PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^6 dB with a maximum of PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^7 dB, and an average MS-SSIM gain of approximately PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^8 dB with a maximum of PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^9 dB.

RVQ-based neural audio codecs: In "Improving Test-Time Performance of RVQ-based Neural Codecs", quantize-then-rectify takes the form of a test-time beam search over RVQ code combinations, replacing greedy level-wise code selection with a search that keeps multiple candidate sums and prunes them by latent-space error PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^ (&&&^^^^5\5^^^^&&&). Greedy RVQ is recovered when the beam width and per-candidate expansion are both PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^. On LibriTTS with EnCodec at PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^ kbps, moving from PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5query5^^^^ to PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5\5^^^^ reduces latent quantization error from PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)55^^^^ to PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^6, improves PESQ from PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^7 to PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^8, and raises SI-SNR from PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^9 to PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5query5^^^^.

Quantized compressed sensing by ReLUs: In the treatment associated here with (&&&^^^^5query5^^^^&&&), the quantize-then-rectify step is a convex reconstruction program built from rectified linear penalties after one-bit or uniform multi-bit quantization with dither. For one-bit measurements, the functional is

PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^

and reconstruction is obtained by minimizing PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5max_results5^^^^ over a convex low-complexity set PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5query5^^^^. The paper proves near-optimal uniform recovery guarantees under isotropic, symmetric, PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5\5^^^^-subgaussian sensing, with explicit regimes for additive noise and adversarial bit corruptions.

6. Scope, limitations, and recurring misconceptions

A recurring misconception is that rectification necessarily requires retraining the full model or changing the compressed representation itself. The surveyed instantiations show otherwise. In efficient VQ-VAE training, the heavy VAE encoder and decoder are frozen and only the latent quantizer and rectifier are trained; in latent-space image compression, the bitstream and entropy model are unchanged because only PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results55^^^^ is entropy-coded; in RVQ-based neural codecs, the codebooks and decoder remain unchanged and rectification is purely a test-time search procedure (&&&^^^^5query5^^^^&&&, Luo et al., 2024, &&&^^^^5\5^^^^&&&).

Another misconception is that ReVQ always rectifies the same error signal. The cases above show multiple targets: latent MSE in VQ-VAE conversion and image compression, inner-product bias in TurboQuant, weight perturbation in LLM PTQ, search suboptimality in RVQ codecs, and measurement inconsistency in compressed sensing (&&&^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^&&&, &&&^^^^5max_results5^^^^&&&, &&&^^^^5query5^^^^&&&). This suggests that ReVQ is best understood as a placement principle—put the corrective operator after the discrete bottleneck—rather than as a fixed architecture.

The limitations are domain-specific. For VQ-VAE training, the method achieves strong quality at PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^6 tokens but does not reach ultra-short sequences such as PRESERVED_PLACEHOLDER_^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5max_results5^^^^7 tokens, and the paper attributes this to rectifier design and the exponential growth in required codebook size as tokens decrease; it also states that, for a fixed VAE architecture, there is an inherent upper bound on achievable compression without retraining the decoder (&&&^^^^5query5^^^^&&&). TurboQuant depends on the high-dimensional near-independence of rotated coordinates and notes reduced efficacy in very low dimensions or under extremely heavy-tailed inputs (&&&^^^^5id:(Zhang et al., 14 Jul 2025) OR id:(Zandieh et al., 28 Apr 2025) OR id:(Cho et al., 2 Feb 2026) OR id:(Jung et al., 2019) OR id:(Kim et al., 23 Sep 2025) OR id:(Luo et al., 2024)5^^^^&&&). The SRR analysis argues that allocating the entire low-rank budget to post-quantization correction can be suboptimal when dominant activation-scaled directions should instead be preserved before quantization (&&&^^^^5max_results5^^^^&&&). The audio beam-search variant introduces a compute-latency trade-off, especially on CPU, and the compressed sensing guarantees depend on explicit assumptions on sensing distributions, dither, convex priors, and corruption regimes (&&&^^^^5\5^^^^&&&, &&&^^^^5query5^^^^&&&).

Within this landscape, the contribution of "Quantize-then-Rectify: Efficient VQ-VAE Training" is to make the principle operational for visual tokenizers under a particularly stringent efficiency target: exploit a pre-trained VAE’s empirical tolerance to latent perturbation, quantize only up to that tolerance, and use a small post rectifier to recover the remaining error, thereby obtaining competitive reconstruction with orders-of-magnitude less compute (&&&^^^^5query5^^^^&&&).