Iterative TARFlow (iTARFlow)

• iTARFlow is a likelihood-trained normalizing flow model that integrates autoregressive generation with an iterative denoising phase over a range of Gaussian noise levels.
• It resolves the noise dilemma by employing multi-noise training and a two-phase sampling process that starts with noisy initialization followed by gradient-driven refinement.
• The iterative framework of iTARFlow has inspired diverse applications in image generation, alignment flows, inverse problems, radar scene estimation, and continuous-space language modeling.

Iterative TARFlow (iTARFlow) is most directly the model introduced in "Normalizing Flows with Iterative Denoising," where a Transformer Autoregressive Flow is trained with exact maximum likelihood over a continuum of Gaussian noise levels and sampled by combining autoregressive generation with a short iterative denoising procedure driven by the model’s own likelihood gradients (Chen et al., 21 Apr 2026). In the broader literature, however, the label has not remained fully standardized: some papers cite iTARFlow only as prior art, while others use the term as an explanatory shorthand for distinct iterative flow procedures in alignment, inverse problems, radar scene flow, continuous-space language modeling, or accelerated TarFlow inversion (Shen et al., 8 May 2026, Zhou et al., 2021, Adler et al., 2018, Fu et al., 18 May 2026, Zhang et al., 1 Jul 2025, Liu et al., 19 May 2025). This suggests that iTARFlow now denotes a cluster of related but non-identical ideas centered on iterative refinement within TARFlow-like or flow-based systems.

1. Naming, scope, and terminological status

Within the supplied literature, the explicit model name "iterative TARFlow (iTARFlow)" is introduced for image generation in "Normalizing Flows with Iterative Denoising" (Chen et al., 21 Apr 2026). That paper defines iTARFlow as a likelihood-trained normalizing flow that performs autoregressive generation at a nonzero noise level and then applies an iterative denoising phase inspired by diffusion-style methods, while preserving end-to-end exact maximum-likelihood training.

Other appearances of the term are narrower or editorial. "STARFlow2: Bridging LLMs and Normalizing Flows for Unified Multimodal Generation" states that iTARFlow is not defined there and that the paper cites it only once as the source of a "multi-noise training strategy" for visual generation; STARFlow2’s own sampling remains single-pass and purely causal (Shen et al., 8 May 2026). "Iterative Alignment Flows" uses the prompt-aligned label iTARFlow for an iterative barycenter-based alignment flow called INB in the paper (Zhou et al., 2021). "Task adapted reconstruction for inverse problems" does not use the name in the original paper, but the supplied notes define iTARFlow as the iterative version of task-adapted reconstruction with an unrolled reconstruction module and a jointly trained task head (Adler et al., 2018). "Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation" uses IterFlow as the official name and refers to it as iTARFlow only to align with the prompt (Fu et al., 18 May 2026). "Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows" likewise does not define iTARFlow explicitly, but the supplied notes use the term for TarFlowLM’s hierarchical multi-pass generation variant (Zhang et al., 1 Jul 2025). "Accelerate TarFlow Sampling with GS-Jacobi Iteration" presents an implementation-oriented guide to iTARFlow as GS-Jacobi-accelerated inverse sampling for TarFlow (Liu et al., 19 May 2025).

Source	Status of the term	Meaning attached to iTARFlow
(Chen et al., 21 Apr 2026)	Explicit model name	Multi-noise TARFlow with iterative denoising
(Shen et al., 8 May 2026)	Cited, not defined	Source of a "multi-noise training strategy"
(Zhou et al., 2021)	Editorial shorthand	Iterative barycenter-based alignment flow
(Adler et al., 2018)	Editorial shorthand	Unrolled task-adapted reconstruction flow
(Fu et al., 18 May 2026)	Prompt-aligned alias	IterFlow for weakly supervised radar scene flow
(Zhang et al., 1 Jul 2025)	Editorial shorthand	Hierarchical multi-pass TarFlowLM generation
(Liu et al., 19 May 2025)	Implementation-oriented usage	GS-Jacobi fixed-point inversion for TarFlow

2. Core generative formulation in "Normalizing Flows with Iterative Denoising"

In its canonical generative sense, iTARFlow is a normalizing-flow model built on TARFlow, an autoregressive normalizing flow with a causal Transformer backbone (Chen et al., 21 Apr 2026). A normalizing flow defines an invertible map $f_\theta:\mathbb{R}^D\to\mathbb{R}^D$ from data $x$ to latent $z=f_\theta(x)$ with log-likelihood

$$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$

With a standard normal prior, $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$ (Chen et al., 21 Apr 2026).

TARFlow instantiates $f_\theta$ as a composition of causal Transformer-based autoregressive flow layers,

$f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$

with affine forward and inverse transforms at layer $\ell$ and token $n$:

$$x^{\ell+1}_n=\frac{x^\ell_n-\mu^\ell_\theta(x^\ell_{<n})}{\sigma^\ell_\theta(x^\ell_{<n})}, \qquad x^\ell_n=\mu^\ell_\theta(x^\ell_{<n})+\sigma^\ell_\theta(x^\ell_{<n})x^{\ell+1}_n.$$

Because the conditioning is causal, the Jacobian is triangular, and the per-sample log-likelihood reduces to

$x$0

with inter-layer permutations used to improve expressiveness without additional log-determinant cost (Chen et al., 21 Apr 2026).

The principal motivation for iTARFlow is the "noise dilemma" in flow training. TARFlow had already shown that additive Gaussian noise can stabilize optimization and improve sampling, but a single fixed noise level $x$1 is problematic: small $x$2 yields overly sharp, texture-heavy, less coherent images and training instability, whereas large $x$3 improves global structure but produces overly smooth, blurry samples after single-shot self-denoising via Tweedie’s lemma (Chen et al., 21 Apr 2026). iTARFlow resolves this by amortizing the same autoregressive flow across a range of noise scales and then refining samples iteratively during inference.

3. Multi-noise likelihood training and two-phase sampling

The defining training objective of iTARFlow is exact maximum likelihood over noisy samples $x$4, with $x$5 and $x$6 (Chen et al., 21 Apr 2026). The objective is

$x$7

and the paper uses the simple noise-dependent weight $x$8 for stability. The practical settings reported are $x$9 for all experiments and $z=f_\theta(x)$0 for ImageNet-64 in pixel space, $z=f_\theta(x)$1 for ImageNet-128 in pixel space, $z=f_\theta(x)$2 for ImageNet-256 in pixel space, and $z=f_\theta(x)$3 for ImageNet-256 in latent space (Chen et al., 21 Apr 2026).

Sampling is explicitly split into two phases. In phase 1, autoregressive flow generation is performed at a single noise level $z=f_\theta(x)$4. One samples $z=f_\theta(x)$5 and inverts the flow autoregressively to obtain $z=f_\theta(x)$6. Classifier-free guidance is applied patch-wise only in the first causal Transformer layer during this phase (Chen et al., 21 Apr 2026).

In phase 2, the noisy sample is denoised iteratively and fully in parallel by integrating the ODE

$z=f_\theta(x)$7

for $z=f_\theta(x)$8 (Chen et al., 21 Apr 2026). The score $z=f_\theta(x)$9 is obtained by automatic differentiation through the same likelihood-trained normalizing flow; no additional denoiser network is introduced. The paper uses a simple linear discretization over $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$0 steps, reports that as few as $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$1 steps produce high-quality images, and finds that $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$2 steps suffices to reach the best FID (Chen et al., 21 Apr 2026). No classifier-free guidance is used during denoising because it adds cost without improving quality.

This design differs from diffusion models in a precise way. iTARFlow does not train a reverse SDE or ODE and does not optimize a score-matching objective; rather, the score used at inference is the gradient of a normalized density learned by exact maximum likelihood (Chen et al., 21 Apr 2026). The resulting system is therefore diffusion-inspired at sampling time but remains a likelihood-trained normalizing flow during optimization.

4. Architecture, empirical behavior, and reported performance

The backbone architecture in the generative iTARFlow paper remains TARFlow with time conditioning (Chen et al., 21 Apr 2026). The reported configuration uses four flow blocks for all model sizes, patch size equal to $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$3 in pixel space, and Fourier features plus a $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$4 embedding for time conditioning. The paper summarizes model sizes as follows: Small uses attention blocks $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$5 with channels $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$6; Big uses $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$7 with channels $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$8; Large uses $$\log p_\theta(x)=\log p_{\text{prior}}(f_\theta(x))+\log\left|\det\frac{\partial f_\theta(x)}{\partial x}\right|.$$9 with channels $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$0; XL uses $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$1 with channels $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$2. For latent ImageNet-256, SD-VAE is used as encoder and decoder (Chen et al., 21 Apr 2026).

Quantitatively, the paper reports substantial FID improvements over TARFlow and strong competitiveness relative to diffusion baselines across ImageNet resolutions (Chen et al., 21 Apr 2026).

Setting	Model	Reported FID
ImageNet-64 pixel	iTARFlow-S	2.05
ImageNet-64 pixel	iTARFlow-B	1.68
ImageNet-128 pixel	iTARFlow-L	2.44
ImageNet-256 pixel	iTARFlow-XL	3.32
ImageNet-256 latent	iTARFlow-B + SD-VAE	2.32

Several ablations clarify where the gains originate. Larger $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$3 generally improves performance, especially at higher resolution, but too small a maximum noise scale reintroduces instability and texture-heavy artifacts (Chen et al., 21 Apr 2026). Replacing TARFlow’s original self-denoiser with the iTARFlow iterative denoiser markedly improves FID at high $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$4; on ImageNet-64 with $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$5, TARFlow-S plus self denoiser yields FID $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$6, whereas TARFlow-S plus the iTARFlow iterative denoiser yields FID $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$7, nearly matching iTARFlow-S at $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$8 (Chen et al., 21 Apr 2026). In latent ImageNet-256, the paper also reports that iTARFlow’s own iterative denoiser performs comparably to a DiT-style iterative denoiser when applied to iTARFlow samples (Chen et al., 21 Apr 2026).

The paper identifies two recurring artifact types in $$\log p_{\text{prior}}(z)=-\frac{1}{2}\|z\|_2^2+\text{const}$$9k ImageNet-256 pixel samples: collapsed background, in which the entire background is rendered black, and low-frequency blur with a characteristic distortion concentrated in the bottom-right region, often the first continuous token (Chen et al., 21 Apr 2026). The proposed explanations are, respectively, over-strong classifier-free guidance with large patch size and the difficulty of generating the first autoregressive token without context. The paper also notes a practical limitation: auto-differentiation through the full normalizing flow during denoising incurs nontrivial GPU memory cost (Chen et al., 21 Apr 2026).

5. Relation to TARFlow, STARFlow, and STARFlow2

iTARFlow is best understood against the progression from TARFlow to STARFlow and STARFlow2. "STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis" presents TARFlow as a Transformer Autoregressive Flow and emphasizes deep-shallow flow design, latent-space modeling in pretrained autoencoders, and a new guidance algorithm, while remaining an end-to-end normalizing flow with exact maximum likelihood training (Gu et al., 6 Jun 2025). The paper explicitly states that it does not define an algorithm called "Iterative TARFlow (iTARFlow)," although it notes iteration-like behavior in the deep-shallow stack and in training-free inpainting with repeated clamping (Gu et al., 6 Jun 2025).

The official iTARFlow of (Chen et al., 21 Apr 2026) can therefore be viewed as a direct extension of TARFlow’s generative program: it keeps the autoregressive flow backbone and exact likelihood training but adds multi-noise conditioning and iterative denoising at inference. By contrast, STARFlow2 places TARFlow inside a unified multimodal architecture. STARFlow2 defines TARFlow as an autoregressive normalizing flow parameterized by a causal Transformer, with per-position affine parameters for continuous latents predicted under the same causal mask and KV-cache as text tokens (Shen et al., 8 May 2026). Its deep-shallow composition uses visual-only shallow AF blocks with alternating scan directions followed by a deep multimodal TARFlow block, and its Pretzel architecture vertically interleaves a frozen Qwen2.5-VL-7B-Instruct stream with a TARFlow stream via zero-initialized residual skip connections (Shen et al., 8 May 2026).

Crucially, STARFlow2 states that iTARFlow is not defined there. The paper cites iTARFlow only once as the source of a "multi-noise training strategy" and explicitly says that the iterative behavior in STARFlow2 comes from deep-shallow TARFlow composition and alternating scan directions in shallow autoregressive flow blocks, not from an iTARFlow-specific multi-pass refinement loop; sampling remains single-pass and purely causal (Shen et al., 8 May 2026). This distinction is important because it separates true iterative denoising, as in (Chen et al., 21 Apr 2026), from architectures that are structurally multi-stage yet still sampled in one left-to-right causal pass.

6. Other domain-specific uses and extensions of the label

Outside image generation, the supplied literature applies the label iTARFlow to several conceptually distinct iterative systems. In "Iterative Alignment Flows," the term refers to an iterative barycenter-based alignment flow that jointly aligns $f_\theta$0 empirical distributions into a shared latent space by alternating between critic maximization over projection directions and exact closed-form univariate optimal-transport alignment maps lifted back to $f_\theta$1 (Zhou et al., 2021). Each iteration solves a variational sliced-Wasserstein surrogate and then composes rank-1 invertible transports that move projected marginals to their Wasserstein barycenter, yielding a deep and expressive alignment flow without adversarial learning (Zhou et al., 2021).

In "Task adapted reconstruction for inverse problems," iTARFlow denotes an unrolled task-adapted reconstruction pipeline in which a reconstruction estimator $f_\theta$2 and a task estimator $f_\theta$3 are trained jointly under a differentiable joint loss (Adler et al., 2018). The iterative part is the learned reconstruction module, implemented as learned gradient descent or learned primal-dual updates involving the forward operator $f_\theta$4 and its adjoint $f_\theta$5 at each step. The flow is therefore the end-to-end differentiable composition

$f_\theta$6

with gradients backpropagated through all unrolled iterations and the task head (Adler et al., 2018).

In "Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation," the official method name is IterFlow, and the supplied notes align it with iTARFlow as "Iterative Task-specific Radar Flow" (Fu et al., 18 May 2026). There the system iteratively refines radar scene flow over $f_\theta$7 iterations using warping, ball-query local correlation, GRU-based recurrent updates, and three training losses only:

$f_\theta$8

The method uses only radar at inference and uses images plus odometry as weak supervision during training; on View-of-Delft it reports EPE $f_\theta$9 m, AccS $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$0, AccR $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$1, RNE $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$2 m, MRNE $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$3 m, and SRNE $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$4 m (Fu et al., 18 May 2026).

In "Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows," the supplied notes use iTARFlow for TarFlowLM’s hierarchical multi-pass generation variant (Zhang et al., 1 Jul 2025). TarFlowLM maps discrete tokens into continuous latents through a VAE bridge and models the latent prior with stacked Transformer-based autoregressive normalizing flows using alternating left-to-right and right-to-left directions, mixture-based couplings, block-wise generation with flexible patch size, and intermediate decoding after partial inversion of the flow stack (Zhang et al., 1 Jul 2025). The resulting multi-pass structure enables coarse-to-fine refinement and text editing in continuous latent space.

Finally, "Accelerate TarFlow Sampling with GS-Jacobi Iteration" uses iTARFlow to describe accelerated inverse sampling for TarFlow as a fixed-point problem (Liu et al., 19 May 2025). The inverse relation

$f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$5

is solved by nonlinear Jacobi or GS-Jacobi iteration rather than strictly sequential ancestral inversion. The paper introduces the Initial Guessing Metric (IGM) to choose between initializing with $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$6 or $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$7, and the Convergence Ranking Metric (CRM) to identify "tough" blocks that require GS partitioning (Liu et al., 19 May 2025). Reported speedups are $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$8 on Img128cond, $f=f_L\circ f_{L-1}\circ\cdots\circ f_1,$9 on AFHQ, $\ell$0 on Img64uncond, and $\ell$1 on Img64cond without degrading FID scores or sample quality (Liu et al., 19 May 2025).

Taken together, these usages suggest that iTARFlow is not a single fixed algorithmic object across the literature. In its narrow and official sense, it is the multi-noise, likelihood-trained, iteratively denoised generative TARFlow of (Chen et al., 21 Apr 2026). In a broader descriptive sense, it has become a label for iterative refinement schemes built on flow models, autoregressive transformations, or task-specific transport procedures across several research domains.