reAR: Enhancing Visual AR Consistency

• The paper reAR introduces a plug-and-play consistency regularization that mitigates generator-tokenizer mismatch in autoregressive image synthesis.
• It employs noisy context training and embedding alignment via an auxiliary MLP to ensure robust token prediction and decoding.
• Empirical results on ImageNet show significant FID and IS improvements, narrowing the gap with diffusion models while retaining fast AR sampling.

Searching arXiv for the specific reAR paper and closely related visual autoregressive generation context. reAR is a training strategy for visual autoregressive (AR) generation that addresses what it identifies as a central failure mode of token-based image synthesis: generator-tokenizer inconsistency. In visual AR pipelines, images are first converted into discrete token sequences by a tokenizer, and a causal transformer is then trained to predict the next token from prior tokens. The reAR formulation argues that the longstanding performance gap between visual AR models and diffusion models is not explained solely by tokenizer design or rasterization order; rather, AR-generated token sequences may be poorly decoded by the tokenizer, even when next-token prediction is locally accurate. To address this, reAR introduces a plug-and-play consistency regularization objective that trains the generator to operate under noisy contexts and to align its hidden states with tokenizer codebook embeddings, while leaving the tokenizer, generation order, inference pipeline, and external model dependencies unchanged (He et al., 6 Oct 2025).

1. Problem setting and conceptual basis

Visual AR models inherit their basic formulation from language modeling: a tokenizer maps an image to a sequence of discrete symbols, and a transformer predicts tokens autoregressively. The paper situates this paradigm as a promising route toward unifying vision and LLMs, but notes that its image-generation quality has remained suboptimal relative to diffusion models. Prior explanations had emphasized tokenizer limitations and rasterization ordering; reAR instead reframes the bottleneck as a mismatch between the generator’s token dynamics and the tokenizer’s decoder geometry (He et al., 6 Oct 2025).

The central notion is generator-tokenizer inconsistency: the AR model may generate token sequences that are syntactically acceptable for next-token prediction yet are not well decoded by the tokenizer. The paper identifies two specific manifestations. The first is amplified exposure bias. Under teacher forcing, training conditions always provide ground-truth context, whereas inference conditions expose the model to its own earlier token errors; in vision, such early errors can cascade into contexts that are unfamiliar to the tokenizer, producing visible artifacts. The second is embedding unawareness. The generator is trained on discrete token indices, but image reconstruction depends on the tokenizer’s codebook embeddings. The paper reports that, under the same correct token ratio, embedding-aware tokens reconstruct much higher-quality images than merely token-correct sequences, indicating that token identity alone is an incomplete training signal for visual AR generation (He et al., 6 Oct 2025).

This diagnosis changes the explanatory emphasis of visual AR research. Rather than treating tokenization and ordering as the only major design levers, reAR treats the relationship between predicted tokens and decoder embeddings as a first-class optimization target.

2. Training objective and regularization mechanism

reAR adds two coupled training components to standard autoregressive learning: noisy context regularization and codebook embedding regularization. The first component perturbs the conditioning context during training so that the generator learns to predict the next token when some previous tokens are corrupted. The paper defines the noisy token sequence as

$$\tilde{\mathbf{x}}_i = (1 - b_i)\,\mathbf{x}_i + b_i\,\mathbf{u}_i,\quad b_i \sim \mathrm{Bernoulli}(\epsilon),\quad \mathbf{u}_i \sim \mathrm{Uniform}(\{1,\dots,K\}),$$

and the corresponding AR loss as

$$\mathcal{L}'_{\mathrm{AR}}(\theta) = -\mathbb{E}_{\mathbf{x}, \tilde{\mathbf{x}}, \epsilon} \sum_{i=1}^{N} \log p_\theta(\mathbf{x}_i \mid \tilde{\mathbf{x}}_{1:i-1}).$$

The corruption rate $\epsilon$ is annealed during training. The paper explicitly contrasts this mechanism with scheduled sampling, stating that noisy-context training preserves full parallelism and thus remains efficient (He et al., 6 Oct 2025).

The second component aligns the generator’s hidden representations with the tokenizer’s embedding space. A small MLP head projects transformer features into the codebook-embedding space, and the model is trained to recover the visual embedding of the current token and predict the embedding of the target token under the noisy context. The regularization term is

$$\mathcal{L}_{\mathrm{re}}(\theta,\phi) = \mathbb{E}_{\mathbf{x}, \tilde{\mathbf{x}}, \epsilon} \sum_{i=1}^{N-1} \Big[ d\!\left(h_\phi^{i}(\mathbf{w}^{l}_\theta(\tilde{\mathbf{x}})),\, \mathbf{z}_{\mathbf{x}_i}\right) + d\!\left(h_\phi^{i}(\mathbf{w}^{l'}_\theta(\tilde{\mathbf{x}})),\, \mathbf{z}_{\mathbf{x}_{i+1}}\right) \Big],$$

where $d$ is cosine distance, $h_\phi^i$ is the projection MLP for token $i$, and regularization is applied at an early layer and a deep layer of the transformer (He et al., 6 Oct 2025).

The full objective is

$$\mathcal{L}_{\mathrm{reAR}}(\theta,\phi) = \mathcal{L}'_{\mathrm{AR}}(\theta) + \lambda\,\mathcal{L}_{\mathrm{re}}(\theta,\phi),$$

with $\lambda$ the regularization weight. The paper characterizes the resulting method as a purely training-time modification: it requires no changes to the tokenizer, generation order, inference pipeline, or external models (He et al., 6 Oct 2025).

3. Layerwise alignment and generator-tokenizer consistency

A notable design feature of reAR is that it does not treat embedding alignment as a uniform penalty across the network. The paper states that embedding regularization is most effective when applied to the current token at an early layer and to the next token at a deep layer. This introduces a layerwise division of labor: shallow features are encouraged to encode the present codebook state, while deeper features are shaped toward forward prediction in the tokenizer’s embedding geometry (He et al., 6 Oct 2025).

This design directly addresses the two previously identified pathologies. Noisy context regularization mitigates exposure bias by making the generator robust to corrupted histories that resemble inference-time errors. Embedding regularization addresses embedding unawareness by forcing the causal transformer to represent tokens in a form that remains decodable by the tokenizer. Because the reconstruction process is ultimately driven by codebook embeddings rather than raw token identities, this alignment objective targets a decoder-relevant representation rather than merely improving discrete classification over token indices.

The paper reports that the two components are synergistic: removing either noisy context or embedding regularization degrades FID. Annealed noise performs better than fixed-ratio noise, and reAR improves generalization and robustness to noisy inputs. The method therefore functions as a consistency bridge between the autoregressive generator and the tokenizer decoder rather than as a tokenizer replacement or an alternative sampling algorithm (He et al., 6 Oct 2025).

4. Empirical performance on ImageNet

The principal evaluation is class-conditional image generation on ImageNet-1K at $256\times256$. The paper reports substantial improvements over vanilla visual AR baselines and competitive performance relative to large diffusion systems (He et al., 6 Oct 2025).

Model/setup	gFID	IS
AR-L, standard VQGAN tokenizer + raster scan, 461M params	3.02	256.2
reAR-L, same tokenizer/setup, 461M params	1.86	316.9
reAR-B-AliTok, 177M params	1.42	—

With a standard rasterization-based tokenizer, reAR reduces gFID from 3.02 to 1.86 and improves IS to 316.9. The paper describes this as the largest FID reduction reported in visual AR. When paired with advanced tokenizers, reAR achieves gFID 1.42 with 177M parameters, matching the performance of larger state-of-the-art diffusion models with 675M parameters (He et al., 6 Oct 2025).

The improvements are not restricted to one tokenizer family. The paper states that reAR improves both bidirectional and unidirectional tokenizers, including VQGAN, TiTok, and AliTok. Reported examples include TiTok improving from FID 4.45 under AR to 4.01 under reAR, and AliTok improving from 1.50 to 1.42 (He et al., 6 Oct 2025). The same summary also notes that scaling up the AR model size lowers FID further and that the scaling trend matches or exceeds prior AR models.

These results are important because they isolate the improvement to training rather than to architecture replacement. The tokenizer and inference pipeline remain fixed, so the reported gains are attributable to the consistency regularization itself.

5. Relation to diffusion models and multimodal autoregression

The paper positions reAR as a mechanism for closing the gap between visual AR and diffusion models without abandoning the autoregressive framework. In the reported ImageNet results, reAR matches diffusion-level performance with substantially fewer parameters in some settings, while retaining the operational advantages of autoregressive decoding (He et al., 6 Oct 2025).

An additional claim concerns efficiency at inference. Because reAR does not alter the AR architecture or sampling algorithm, the model preserves the speed characteristics of standard autoregressive generation, including the use of KV caching. The paper states that sampling remains fast and that throughput matches or exceeds state-of-the-art alternatives such as diffusion and MAR systems (He et al., 6 Oct 2025).

This has a broader systems implication for multimodal modeling. The AR paradigm is already central to LLMs, and the paper argues that effective high-fidelity token-based image generation strengthens the case for unified vision-LLMs. Since reAR is tokenizer-agnostic, plug-and-play, and does not require changes to the token vocabulary or generation protocol, it can be integrated into multimodal autoregressive stacks without introducing a separate inference-time mechanism. This suggests that some of the historical resistance to visual AR on quality grounds may be mitigated by improving generator-tokenizer compatibility rather than by replacing autoregression as the underlying generative principle.

6. Interpretation and research significance

reAR reinterprets a long-running problem in image generation. Instead of attributing the limitations of visual AR primarily to token order or tokenizer expressivity, it identifies a training mismatch between the discrete next-token objective and the continuous embedding space that governs image decoding. The method’s two-part regularization scheme addresses both sources of inconsistency: corrupted-context robustness and embedding-space alignment (He et al., 6 Oct 2025).

The paper’s experimental profile supports that interpretation. Improvements appear across tokenizer families, both regularization components matter in ablation, and the method preserves the full AR inference pipeline. A plausible implication is that substantial quality gains in visual AR can be obtained without redesigning the tokenizer or adopting a different generation order, provided that training more directly constrains the generator to remain compatible with the decoder’s embedding geometry.

Within the visual generation literature, reAR is therefore best understood not as a new tokenizer or a new sampler, but as a consistency-regularized training framework for autoregressive image models. Its principal contribution is methodological: it inserts decoder-aware structure into AR training while retaining the simplicity and modularity of the original pipeline. That combination—training-time intervention, inference-time continuity, and competitive image-generation metrics—defines its place in the evolution of visual autoregressive modeling (He et al., 6 Oct 2025).