IAR2: Semantic-Detail Autoregressive Image Generation

• IAR2 is an autoregressive image-generation framework that uses a semantic-detail hierarchy and dual codebooks to decouple coarse and fine image representations.
• It leverages a hierarchical autoregressive prediction scheme that first predicts global semantics then details, aligning token prediction with the intrinsic structure of images.
• IAR2 employs progressive attention-guided adaptive classifier-free guidance, achieving state-of-the-art ImageNet FID scores while reducing GPU training resources.

IAR2 is an autoregressive image-generation framework introduced for token-based visual synthesis that explicitly models a semantic-to-detail hierarchy rather than treating visual tokens as a flat vocabulary. In the formulation presented in “IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction,” the method combines a Semantic-Detail Associated Dual Codebook, a Semantic-Detail Autoregressive Prediction scheme, a Local-Context Enhanced Autoregressive Head, and a Progressive Attention-Guided Adaptive CFG mechanism. The paper reports class-conditional ImageNet performance down to a Fréchet Inception Distance (FID) of 1.50 for IAR2-XXL with reject sampling, positioning the model as a state-of-the-art autoregressive generator in that benchmark setting (Yi et al., 8 Oct 2025).

1. Conceptual basis and relation to prior IAR

IAR2 is presented as a successor to a prior method, IAR, that had already attempted to make autoregressive image generation more robust by reorganizing a pretrained visual codebook according to embedding similarity. In that earlier formulation, the guiding idea was that nearby codebook embeddings often decode to visually similar patches, so exact token prediction can be an unnecessarily rigid criterion. The new framework keeps the structural motivation but replaces post hoc codebook rearrangement with a representation and prediction scheme designed around semantic hierarchy from the outset (Yi et al., 8 Oct 2025).

The paper identifies two limitations in the prior IAR formulation. The first is the rigidity of pretrained codebooks. The second is the use of hard, uniform clustering, which can force semantically different embeddings into the same group while also imposing equal cluster sizes. IAR2 addresses these constraints by decoupling patch representation into separate semantic and detail components and then predicting them hierarchically rather than as a single undifferentiated token (Yi et al., 8 Oct 2025).

A central motivation is the mismatch between the structure of images and the structure assumed by many autoregressive models. Images have strong local correlation and a coarse-to-fine organization in which global semantic content constrains admissible local detail. The paper therefore frames IAR2 as an attempt to make tokenization and sequence modeling better aligned with visual structure. It also emphasizes a reconstruction-versus-generation trade-off in single-codebook systems: increasing codebook size tends to improve reconstruction FID while potentially degrading generative FID because prediction becomes harder over a larger vocabulary (Yi et al., 8 Oct 2025).

2. Semantic-Detail Associated Dual Codebook

The core representational change in IAR2 is the introduction of two codebooks: a semantic codebook $\mathcal C_s$ of size $n_1$ and a detail codebook $\mathcal C_d$ of size $n_2$. For an image patch $I_i$, the encoder produces a latent embedding

$e_i = E(I_i).$

The semantic token is obtained by nearest-neighbor lookup in $\mathcal C_s$,

$$q_{i,s} = \underset{q_{i,s}\in [1,n_1]}{\arg\min} \, \| e_i - c_s^{q_{i,s}} \|_2^2,$$

and a residual

$e_{i,res} = e_i - c_s^{q_{i,s}}$

is then quantized against the detail codebook,

$$q_{i,d} = \underset{q_{i,d}\in [1,n_2]}{\arg\min} \, \| e_{i,res} - c_d^{q_{i,d}} \|_2^2.$$

The final quantized latent is

$n_1$0

with reconstruction

$n_1$1

The paper states that this expands representational capacity from linear to polynomial scale, with effective capacity $n_1$2 (Yi et al., 8 Oct 2025).

Training of this tokenizer is explicitly two-stage. In Stage 1, the encoder, decoder, and semantic codebook are trained using a semantic reconstruction objective plus a VQ commitment term, so that the semantic codebook absorbs coarse visual content before detail refinement is introduced. In Stage 2, the detail codebook is added and all components are jointly optimized with an $n_1$3 reconstruction term, a GAN term, and a semantic-detail commitment term. To preserve the semantic role of $n_1$4, the Stage 1 semantic-only objective is interleaved with Stage 2 updates at a 2:1 ratio (Yi et al., 8 Oct 2025).

The default codebook configuration reported in the paper uses 256 semantic codes and 4096 detail codes. In the appendix comparison on ImageNet $n_1$5 with downsampling rate 16, the tokenizer is reported to reach reconstruction metrics of rFID $n_1$6, PSNR $n_1$7, and SSIM $n_1$8. The paper interprets these numbers as evidence that the dual codebook provides both high expressiveness and tractable prediction complexity (Yi et al., 8 Oct 2025).

3. Hierarchical autoregressive prediction and local context

Once tokenized, each patch is represented by a semantic-detail pair $n_1$9. The paper rejects a naive doubled sequence such as $\mathcal C_d$0, arguing that it both doubles sequence length and flattens the intended hierarchy. Instead, IAR2 embeds the semantic and detail tokens separately and fuses them into a single patch representation,

$\mathcal C_d$1

An autoregressive backbone based on LlamaGen then processes the patch-length sequence $\mathcal C_d$2, preserving the original patch count rather than doubling it (Yi et al., 8 Oct 2025).

Prediction is factorized hierarchically. For the next patch, the model first predicts the semantic token and then predicts the detail token conditioned on both context and the semantic prediction: $\mathcal C_d$3 The training objective is a weighted sum of semantic and detail cross-entropies,

$\mathcal C_d$4

The paper reports that $\mathcal C_d$5 gave the best FID in one B-model ablation, while appendix settings vary across scales (Yi et al., 8 Oct 2025).

A further architectural element is the Local-Context Enhanced Autoregressive Head. This head aggregates hidden states from previously generated tokens within a $\mathcal C_d$6 local window, compresses them with a lightweight shared module, concatenates the compressed vectors, and projects them back to hidden size: $\mathcal C_d$7 The resulting local summary is used together with the global transformer state to refine semantic and detail prediction. The exact local window size $\mathcal C_d$8 is not specified in the provided text. The paper presents this head as a means of improving local spatial coherence without modifying the entire backbone (Yi et al., 8 Oct 2025).

4. Conditional generation and Progressive Attention-Guided Adaptive CFG

The main experimental setting for IAR2 is class-conditional ImageNet generation. Conditioning is handled through classifier-free guidance (CFG), but the paper argues that the standard fixed-scale formulation is suboptimal because it applies a uniform guidance strength to all tokens and keeps that strength constant through the generation trajectory (Yi et al., 8 Oct 2025).

Standard CFG is written as

$\mathcal C_d$9

where $n_2$0 is fixed. IAR2 replaces this with Progressive Attention-Guided Adaptive CFG. First, the model derives a token-specific relevance score $n_2$1 from attention weights from image tokens to condition tokens. Second, it uses a linearly increasing base schedule

$n_2$2

where $n_2$3 is the number of image tokens. The final guidance scale becomes

$n_2$4

This makes guidance both spatially adaptive and temporally progressive (Yi et al., 8 Oct 2025).

For IAR2-B, the best reported schedule among the tested settings is $n_2$5 and $n_2$6, yielding FID $n_2$7, Inception Score $n_2$8, Precision $n_2$9, and Recall $I_i$0. The appendix also lists model-specific schedules for 16$I_i$116 and 24$I_i$224 tokenizations. This suggests that PAG-CFG is tuned as an inference-time component rather than treated as a universal fixed rule (Yi et al., 8 Oct 2025).

5. Empirical performance, scaling, and ablations

The principal benchmark is class-conditional ImageNet at $I_i$3, evaluated on 50,000 synthesized images across all 1,000 validation classes. The paper reports results for model scales from IAR2-B at 143M parameters to IAR2-XXL at 1.5B parameters, with experiments at both $I_i$4 and $I_i$5 token resolutions (Yi et al., 8 Oct 2025).

Model	Parameters	FID
DiT-XL/2	675M	2.27
VAR-d30	2.0B	1.92
LlamaGen-XXL	1.4B	2.34
IAR-XXL	1.4B	2.19
IAR2-XXL	1.5B	1.76
IAR2-XXL$I_i$6	1.5B	1.50

The asterisk denotes reject sampling. The paper also emphasizes a resource-efficiency comparison: IAR2-XXL reportedly outperforms VAR while using 32 GPUs for training, whereas VAR required 256 GPUs. No detailed FLOP or wall-clock table is provided in the text excerpt, so the comparison is framed at the level of GPU count rather than full systems profiling (Yi et al., 8 Oct 2025).

Ablations support the contribution of each major module. In the full-module study at 143M parameters and 100 epochs, the baseline without the three core components gives FID $I_i$7, adding semantic-detail association yields $I_i$8, adding semantic-detail association plus local context yields $I_i$9, adding semantic-detail association plus PAG-CFG yields $e_i = E(I_i).$0, and the full model reaches $e_i = E(I_i).$1. A separate prediction-paradigm ablation reports FID $e_i = E(I_i).$2 for alternating prediction on a doubled sequence, $e_i = E(I_i).$3 for grouped sequential prediction on a doubled sequence, $e_i = E(I_i).$4 for fused independent prediction, and $e_i = E(I_i).$5 for fused hierarchical prediction, directly supporting the semantic-first factorization (Yi et al., 8 Oct 2025).

Other ablations further sharpen the design rationale. A codebook-architecture comparison gives FID $e_i = E(I_i).$6 for a single codebook baseline, $e_i = E(I_i).$7 for an unassociated dual codebook, and $e_i = E(I_i).$8 for the associated dual codebook. A local-context comparison gives $e_i = E(I_i).$9 with no local enhancement, $\mathcal C_s$0 with local enhancement but no compression, and $\mathcal C_s$1 with both local enhancement and compression. A semantic-loss-weight sweep gives FID values of $\mathcal C_s$2, $\mathcal C_s$3, $\mathcal C_s$4, $\mathcal C_s$5, and $\mathcal C_s$6 for $\mathcal C_s$7, respectively, indicating a broad optimum around $\mathcal C_s$8 in that setting (Yi et al., 8 Oct 2025).

6. Terminological ambiguity and unrelated usages

The label “IAR2” is unusually prone to collision with unrelated names. In autonomous-driving trajectory forecasting, the related but distinct method is “AIR$\mathcal C_s$9 for Interaction Prediction,” where the name refers to joint confidence prediction over the Cartesian product of anchored motion hypotheses for two interacting agents; that work concerns the 2021 Waymo Interaction Prediction Challenge and is unrelated to visual token generation (Wu et al., 2021).

In accelerator science, “IAR2” does not appear as an official acronym in the paper “Illinois Accelerator Research Center”; the paper consistently uses IARC for the Illinois Accelerator Research Center at Fermilab, and the provided description identifies “IAR2” in that context as most likely a variant, typo, or mistaken rendering rather than a distinct program (Kroc et al., 2017). In solar spectroscopy, the closest naming collision is “IRIS$$q_{i,s} = \underset{q_{i,s}\in [1,n_1]}{\arg\min} \, \| e_i - c_s^{q_{i,s}} \|_2^2,$$0,” a database-driven inversion tool for IRIS raster spectroscopy that recovers thermodynamic quantities and chromospheric integrated radiative losses; it is a fast inversion framework and not an autoregressive visual generator (Dalda et al., 13 Jan 2026). In radio astronomy, “IAR” denotes the Instituto Argentino de Radioastronomía, including the MIA pathfinder work from the IAR Electronics Laboratory, again unrelated to the 2025 image-generation framework (Gonzalez et al., 2023).

Within current arXiv usage, therefore, the most direct and official sense of “IAR2” is the 2025 autoregressive image-generation method defined in “IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction” (Yi et al., 8 Oct 2025).