Visual AutoRegressive Inverse Noise (VARIN)
- VARIN is a framework that combines visual autoregressive modeling with explicit noise inversion to improve image generation and representation quality.
- It encompasses diverse implementations, including DARL for diffusion-based objectives, discrete inversion for text-guided editing, and DeVAR for CT denoising.
- Empirical results demonstrate VARIN’s competitive performance in image quality metrics and efficiency across tasks such as medical imaging and visual editing.
Searching arXiv for papers on “Visual AutoRegressive Inverse Noise” and related formulations. Visual AutoRegressive Inverse Noise (VARIN) denotes a family of methods that combine visual autoregressive modeling with explicit noise inversion or denoising objectives. Across the literature, the term has been used in at least three closely related but non-identical senses: as a patch-level representation-learning framework that replaces autoregressive regression with a diffusion-style inverse-noise loss in a decoder-only Vision Transformer (Li et al., 2024); as a discrete noise inversion technique for next-scale autoregressive text-based image editing in visual autoregressive token models (Dao et al., 2 Sep 2025); and, in the context of low-dose computed tomography denoising, as a paradigm in which multi-scale visual autoregression, residual inverse-noise recovery, and hybrid decoding are combined to map low-dose CT (LDCT) to normal-dose CT (NDCT) (Zhang et al., 26 Jun 2026). A broader antecedent is noise-conditional maximum likelihood estimation for autoregressive generative modeling, which conditions autoregressive likelihoods on a continuum of noise levels and couples exact likelihood training with score-based refinement (Li et al., 2022). Taken together, these works define VARIN less as a single architecture than as an autoregressive design pattern in which latent or observable variables are generated sequentially while noise is either modeled, inverted, or removed explicitly.
1. Terminological scope and conceptual definition
The common structure underlying VARIN is the combination of two ingredients. The first is a visual autoregressive factorization, typically over image patches or over multi-scale discrete token maps. The second is an inverse-noise mechanism, which may take the form of a diffusion loss, a noise-conditional likelihood objective, an explicit inversion of sampling noise, or a residual denoiser that reconstructs information discarded by quantization.
In "Denoising Autoregressive Representation Learning" (Li et al., 2024), VARIN corresponds to what that work calls DARL: a decoder-only Transformer predicts image patches autoregressively, and image generation ability is enhanced by replacing the MSE loss with a diffusion objective using a denoising patch decoder. In "Discrete Noise Inversion for Next-scale Autoregressive Text-based Image Editing" (Dao et al., 2 Sep 2025), VARIN is defined as a noise inversion-based editing technique for visual autoregressive models, built around a pseudo-inverse for argmax sampling called Location-aware Argmax Inversion (LAI). In "DeVAR: Low-Dose CT Denoising via Visual Autoregressive Modeling" (Zhang et al., 26 Jun 2026), the term appears as an explanatory label for a paradigm in which multi-scale visual autoregression is used to reconstruct NDCT token maps from LDCT prefix tokens, while a residual refiner restores inverse-noise components lost through quantization.
This suggests that VARIN is best understood as an umbrella concept rather than a uniquely fixed method. A plausible implication is that the defining feature is not any single backbone, but the explicit treatment of noise within an autoregressive generative process.
2. Core autoregressive formulations
A central formulation appears in autoregressive representation learning over image patches. In the DARL formulation, the model is a decoder-only Vision Transformer with causal masking, non-overlapping image patches, and a prepended start-of-sequence token. The plain autoregressive objective is
$L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$
where the -th patch is predicted from previous patches (Li et al., 2024). The same work also uses a denoising patch decoder and a DDPM-style objective,
$L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$
The paper states that the MSE and diffusion losses are not summed; each is used as an alternative pre-training objective (Li et al., 2024).
A different autoregressive factorization appears in DeVAR for LDCT denoising. There, the LDCT-to-NDCT mapping is cast as a conditional visual autoregressive generation problem over multi-scale token maps:
$p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$
At each scale, the transformer conditions on previously generated NDCT tokens, fixed LDCT prefix tokens, and a learned task token to predict the next-scale discrete map (Zhang et al., 26 Jun 2026).
In text-based image editing, the autoregressive decomposition is over increasing-resolution discrete token maps:
The image is encoded by a VAR-VAE encoder into discrete token maps, and editing modifies later scales under a target prompt while preserving earlier content or reusing inverted noise (Dao et al., 2 Sep 2025).
The older NCML framework generalizes autoregressive modeling to noisy observations. It defines a noise-conditioned factorization
and optimizes expected negative log-likelihood over noise levels sampled from a prior (Li et al., 2022). This establishes an important precursor: inverse-noise structure can be integrated at the likelihood level rather than only in latent-space denoisers or editing-time inversion.
3. Inverse-noise mechanisms
The inverse-noise component is instantiated differently across the main VARIN formulations.
In DARL, inverse noise is implemented through a diffusion objective applied to patches. Following DDPM, a clean patch $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$0 is corrupted as
$L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$1
and the model learns to predict $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$2 from the noisy patch and timestep. The paper reports that the optimal schedule differs significantly from schedules used in standard image diffusion models and samples the noise level $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$3 directly from a Beta distribution $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$4 (Li et al., 2024).
In DeVAR, inverse noise arises because vector quantization discards high-frequency information. NDCT images are encoded into a continuous latent $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$5, quantized to discrete code indices at each scale via a learned codebook, and reconstructed into a discrete-only latent $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$6. The quantization error motivates a residual refiner, which models
$L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$7
with a lightweight conditional diffusion or MLP network $L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$8. Its training loss is
$L_{\text{AR} = \mathbb{E}_{x\sim\mathcal{D} \sum_{t=1}^T \bigl\|\,x_t \;-\; f_\theta(x_{<t})\bigr\|^2$9
At inference, the estimated residual 0 is added back to 1 before decoding (Zhang et al., 26 Jun 2026).
In discrete editing VARIN, inverse noise is literal inversion of the Gumbel noise underlying argmax sampling. The model assumes sampling by the Gumbel-max trick and, given logits 2 and ground-truth labels 3, defines a pseudo-inverse 4 that yields 5 satisfying 6. The inverted noise is then 7 (Dao et al., 2 Sep 2025). The truncation parameter 8 is used to balance the similarity of 9 to standard Gumbel noise against editing stability.
In NCML, inverse noise is not explicit inversion of a realization, but the model is trained on Gaussian-corrupted data at multiple noise levels and then used with a score-based reverse process. This couples autoregressive likelihood estimation with a reverse-SDE style denoising trajectory (Li et al., 2022).
4. Architectural patterns and training strategies
The architectural diversity of VARIN reflects its status as a paradigm rather than a single model family.
DARL uses a decoder-only Vision Transformer with causal masking and 2D decomposed Rotary Positional Embedding. For MSE training, a single linear layer maps decoder outputs to reconstructed patches. For diffusion training, a small denoising decoder consisting of one Transformer block takes both the backbone output and the noisy patch embedding (Li et al., 2024). At fine-tuning time, causal masking is removed and the final Transformer layer’s output for the last patch token is used as the global image descriptor, with a linear classification head (Li et al., 2024).
DeVAR is built on a multi-scale VQVAE and a transformer that performs next-scale prediction over discrete token maps. Its most distinctive training component is “Dual-Latent Hybrid Training,” which finetunes the decoder $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$0 so that it can invert both the continuous latent $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$1 and the discrete-only reconstruction $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$2. The paper defines
$L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$3
with optional augmentation by a perceptual loss (Zhang et al., 26 Jun 2026). This decoder is then used together with the autoregressive transformer and residual refiner.
The editing-oriented VARIN formulation assumes an existing discrete VAR backbone such as HART or Switti and adds no extra training for editing. Instead, it introduces an inversion phase, which computes inverse noises for all scales in parallel from a source image and source prompt, and an editing phase, in which the model recomputes logits under a target prompt and combines fresh Gumbel noise with the inverted noise via a scale-dependent $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$4 schedule (Dao et al., 2 Sep 2025).
NCML uses a PixelCNN++-style backbone with axial-attention layers and Gaussian Fourier embedding of the noise level $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$5. The same autoregressive network therefore models a family of conditional densities $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$6 instead of a single unconditional density (Li et al., 2022).
5. Application domains and empirical results
The most direct medical-imaging application is LDCT denoising. DeVAR evaluates on two public abdominal-CT benchmarks, Mayo-2016 and Mayo-2020, and reports the best PSNR and RMSE among prior methods, with highly competitive SSIM (Zhang et al., 26 Jun 2026). The paper gives the following examples.
| Dataset | Reported DeVAR result | Comparison stated |
|---|---|---|
| Mayo-2016 | PSNR = 24.54 dB | vs. 24.50 next best |
| Mayo-2016 | SSIM $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$7 | vs. 0.835 for GAN-based and diffusion baselines |
| Mayo-2020 | PSNR = 28.30 dB | vs. 27.39 next best |
| Mayo-2020 | SSIM $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$8 | vs. 0.891 |
The same source states that DeVAR qualitatively recovers subtle vessels and fine textures that CNNs oversmooth and GANs sometimes hallucinate, and that inference remains within clinically acceptable bounds, with inference approximately $L_{\text{diff} =\mathbb{E}_{x_0,\varepsilon,t} \Bigl[\, \bigl\|\, \varepsilon -\varepsilon_\theta(x_t,t) \bigr\|^2 \Bigr].$9–$p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$0 seconds per slice in the reported 4-GPU experiments (Zhang et al., 26 Jun 2026).
In representation learning, DARL reports ImageNet fine-tuning top-1 accuracy after 800-epoch pretraining and 90-epoch fine-tuning. The reported results are 82.7% for ViT-B/16 under AR-MSE and 81.9% under diffusion, 84.7% and 84.9% for ViT-L/16, and 85.5% and 85.9% for ViT-H/16 (Li et al., 2024). The same paper compares these with MAE at 83.6%, 85.9%, and 86.9%, and states that the gap is less than or equal to 1% (Li et al., 2024). On VTAB natural tasks, the diffusion variant achieves 88.7% mean versus 85.9% for a supervised baseline (Li et al., 2024).
For text-guided image editing, VARIN is evaluated on PIE-Bench, which contains 700 images across 9 editing scenarios, using a HART discrete VAR with approximately 14 scales and also experiments on Switti (Dao et al., 2 Sep 2025). The reported hyperparameters include start scale $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$1, truncation $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$2, and a linear decay from $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$3 to $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$4 (Dao et al., 2 Sep 2025). The paper reports the following comparison against training-free baselines:
| Method | Model | Selected reported results |
|---|---|---|
| Regeneration | HART | Struct. 25.56, PSNR 20.45, SSIM 73.31 |
| DICE | Paella | Struct. 11.34, PSNR 27.29, SSIM 89.79 |
| EditAR | LlamaGen | Struct. 39.43, PSNR 21.32, SSIM 75.13 |
| VARIN | HART | Struct. 11.46, PSNR 26.54, SSIM 85.39 |
The paper states that VARIN achieves the best trade-off of structural fidelity and background preservation among discrete methods, that its CLIP scores rival leading continuous-diffusion inversions such as Null-text and PnP Inversion, and that it runs approximately $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$5 faster at approximately $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$6 second per image (Dao et al., 2 Sep 2025).
In autoregressive density modeling, NCML reports 3.32 bits per dimension on ImageNet $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$7 and improves CIFAR-10 FID from 37.50 for Sparse Transformer to 12.09 for the NCML-VP variant (Li et al., 2022). These results are not presented as visual editing or denoising outcomes, but they provide evidence that conditioning autoregressive models on noise levels can improve both robustness and sample quality.
6. Limitations, distinctions, and common misconceptions
A recurrent source of confusion is that VARIN does not denote a single canonical algorithm. In the current literature, the same phrase has been attached to at least three method families with distinct operational meanings: diffusion-style inverse-noise pretraining for autoregressive patch models (Li et al., 2024), discrete Gumbel-noise inversion for editing (Dao et al., 2 Sep 2025), and residual recovery of quantization-discarded detail in conditional CT denoising (Zhang et al., 26 Jun 2026). Any technical use of the term therefore requires attention to context.
Another misconception is that VARIN always combines autoregressive and diffusion losses simultaneously. DARL explicitly states that the MSE and diffusion losses are not summed and are instead used as alternative pre-training objectives (Li et al., 2024). By contrast, DeVAR does combine autoregressive next-scale generation with a residual refiner trained by a diffusion-style loss and a hybrid decoder training strategy (Zhang et al., 26 Jun 2026). These are materially different compositions.
It is also inaccurate to equate all inverse-noise components with standard continuous diffusion. In editing VARIN, the inversion target is the implicit Gumbel noise of argmax sampling, not Gaussian noise in pixel or latent space (Dao et al., 2 Sep 2025). In NCML, the model learns noisy conditional likelihoods and then uses a score-based sampling procedure; this is again a different mechanism (Li et al., 2022).
The reported limitations are correspondingly heterogeneous. DARL notes a remaining small 1% gap to the strongest masked-prediction methods and weaker linear probes, with the decoder-only ViT requiring full fine-tuning (Li et al., 2024). Editing VARIN states that large pose or structural changes and multi-object interactions may violate the linear noise interpolation assumption, causing artifacts or incomplete edits (Dao et al., 2 Sep 2025). DeVAR attributes a core challenge to quantization-induced loss of high-frequency information and addresses it with a residual refiner; this implies that the effectiveness of the method depends on how well that residual pathway restores detail beyond the codebook capacity (Zhang et al., 26 Jun 2026).
7. Research trajectory and outlook
The historical trajectory suggested by the cited works runs from noise-conditioned autoregressive likelihoods, through autoregressive denoising objectives for visual representation learning, to explicit inversion of discrete sampling noise and finally to domain-specific conditional denoising systems. NCML established that autoregressive models can be trained across a continuum of noise levels and then sampled by reverse score refinement (Li et al., 2022). DARL showed that a decoder-only Transformer with a denoising patch decoder can attain representation quality close to state-of-the-art masked prediction models while preserving autoregressive generative structure (Li et al., 2024). Editing VARIN extended the idea of inverse noise into the discrete-token regime by reconstructing Gumbel perturbations consistent with a source image and then reusing them for prompt-guided edits (Dao et al., 2 Sep 2025). DeVAR adapted the general principle to a medical-imaging setting, where visual autoregression captures global-to-local structure and inverse-noise recovery compensates for quantization losses during LDCT denoising (Zhang et al., 26 Jun 2026).
Several future directions are stated explicitly in the literature. DARL points to scaling model and decoder size, hybrid MSE+diffusion curricula, and dynamic patch ordering or latent bottlenecks (Li et al., 2024). Editing VARIN proposes richer pseudo-inverse flows, hybrid attention-based guidance, task-specific tuning of $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$8 and $p\bigl(x_{\mathrm{NDCT}\mid x_{\mathrm{LDCT}\bigr) \;=\; p\bigl(Z_1^{\mathrm{ND},\dots,Z_K^{\mathrm{ND}\mid Z^{\mathrm{LD}\bigr) \;=\; \prod_{s=1}^K p\!\bigl(Z_s^{\mathrm{ND}\;\bigm|\;Z_{<s}^{\mathrm{ND},\,Z^{\mathrm{LD}\bigr)\,.$9 schedules, and extension to other autoregressive frameworks (Dao et al., 2 Sep 2025). DeVAR’s design suggests that residual refinement and hybrid latent decoding may be transferable to other medical inverse problems, although this remains an inference rather than an explicitly reported result (Zhang et al., 26 Jun 2026).
In aggregate, VARIN identifies an important direction in visual generative modeling: preserving the compositional, likelihood-oriented structure of autoregressive generation while introducing explicit mechanisms for handling corruption, uncertainty, or lost detail. The specific instantiation varies by task, but the underlying principle is consistent: sequential generation becomes more effective when the noise process is not merely tolerated, but modeled, inverted, or reconstructed directly.