CoF-T2I: Chain-of-Frame Text-to-Image Synthesis

• CoF-T2I is a text-to-image framework that reinterprets synthesis as a three-frame latent reasoning chain progressing from semantic draft to refined final output.
• It leverages a pretrained video diffusion model with explicit intermediate states to enable stepwise corrections and improved contextual understanding.
• Empirical results demonstrate progressive improvement across frames, achieving superior performance on benchmarks like GenEval and Imagine-Bench.

Searching arXiv for CoF-T2I and closely related work to ground the article. CoF-T2I is a text-to-image generation framework that repurposes a pretrained video generation model as a pure visual reasoner by casting image synthesis as a short Chain-of-Frame (CoF) trajectory rather than a single-pass decoding problem. In this formulation, a prompt is realized through a three-frame sequence that proceeds from coarse semantic draft to intermediate refinement to final image, with the last frame used as the output image. The framework was introduced to exploit the emergent frame-by-frame visual inference behavior observed in large video models, and is trained using CoF-Evol-Instruct, a dataset of prompt-aligned three-frame trajectories designed to model generation from semantics to aesthetics (Tong et al., 15 Jan 2026). The proposal is situated against a broader evaluation context in which text-to-image systems still exhibit substantial weaknesses on visually grounded reasoning tasks, including commonsense-sensitive prompt distinctions (Fu et al., 2024).

1. Conceptual basis

Chain-of-Frame reasoning denotes the spontaneous ability of large video diffusion or flow-matching models to decompose a generation task into a short sequence of coherent “draft → refine → finalize” frames, each functioning as an explicit visual reasoning step (Tong et al., 15 Jan 2026). During training on real video, such models learn how one frame evolves into the next under a causal spatiotemporal prior. When prompted at inference time to generate a short “video,” the model can therefore perform stepwise corrections rather than attempting immediate one-shot synthesis.

CoF-T2I applies this capability to text-to-image generation by reframing image synthesis as a three-frame visual reasoning chain. The intermediate states are explicit and interpretable: the first frame provides a rough semantic draft, the second improves object placement and attributes, and the third polishes fine-grained texture and lighting (Tong et al., 15 Jan 2026). The work characterizes this behavior as analogous to Chain-of-Thought in LLMs, but operating purely in pixel or latent space.

A central motivation is that conventional text-to-image pipelines lack a clearly defined visual reasoning starting point and do not expose interpretable intermediate states. CoF-T2I addresses this by making the intermediate refinement process explicit. This suggests a shift from latent-only inference toward externally inspectable generation trajectories, with the trajectory itself acting as a structured visual computation rather than an incidental by-product.

2. Generation pipeline

CoF-T2I implements text-to-image generation as a three-frame trajectory sampled in the latent space of a video model (Tong et al., 15 Jan 2026). The pipeline begins by prepending a short system prefix to the prompt, for example: “Generate a short refinement chain of the same concept and composition, improving the image step by step.” This prefix conditions the model to interpret the task as iterative refinement rather than unrestricted video continuation.

Generation starts from isotropic Gaussian noise in the video latent space,

$x_0\sim\mathcal{N}(0,I).$

A rectified-flow or diffusion transformer $F_\theta$ then denoises over a continuous schedule $t\in[0,1]$, producing a three-frame latent trajectory $\{z_1,z_2,z_3\}$, formally written as

$z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$

Only the final latent $z_3$ is decoded by the VAE decoder $D$ to produce the output image,

$\hat I = D(z_3).$

The first two latent frames serve as explicit reasoning steps and are discarded after inference (Tong et al., 15 Jan 2026).

This pipeline preserves the composition and concept across frames while encouraging progressive correction. The design is therefore neither a conventional image-editing cascade nor a standard autoregressive storyboard; instead it is a constrained latent refinement chain in which all frames correspond to the same prompt realization. A plausible implication is that CoF-T2I uses temporal inductive bias not to model motion, but to organize successive visual decisions about semantics, structure, and appearance.

3. Architecture and mathematical formulation

CoF-T2I is built on the pretrained video generation backbone Wan2.1-T2V, specifically Wan2.1-T2V-14B, described as a DiT-based flow transformer (Tong et al., 15 Jan 2026). Its architecture comprises a causal video VAE and a transformer-based flow matcher.

The causal video VAE performs spatial downsampling by $8\times$ and temporal downsampling by $4\times$ across a $F_\theta$0 compression schedule. CoF-T2I modifies this arrangement by introducing an independent encoding mode in which each of the three frames is encoded separately by sliding the VAE’s 1-frame context window. This removes temporal compression for the three-frame chain and is explicitly intended to eliminate motion artifacts (Tong et al., 15 Jan 2026). The distinction is consequential because the task is not video synthesis in the ordinary sense; temporal coherence is needed only insofar as it supports semantic refinement.

The transformer-based flow matcher $F_\theta$1 takes as input the noisy latent $F_\theta$2, time $F_\theta$3, a text embedding $F_\theta$4 from a frozen text encoder, and optionally a visual conditioning vector $F_\theta$5 summarizing previous frames. Cross-attention injects $F_\theta$6 and $F_\theta$7 at each self-attention block. Depending on the formulation, the model outputs either a velocity vector $F_\theta$8 in rectified flow or a predicted noise $F_\theta$9 in diffusion form (Tong et al., 15 Jan 2026).

Although the implementation uses rectified flow, the method is also described under diffusion formalism. The forward noising process is given by

$t\in[0,1]$0

and the reverse denoising process by

$t\in[0,1]$1

where $t\in[0,1]$2 denotes conditioning from the text prompt and past-frame latents (Tong et al., 15 Jan 2026).

The simplified denoising objective is

$t\in[0,1]$3

The variational lower bound is stated as

$t\in[0,1]$4

In practice, training uses a continuous flow-matching loss over $t\in[0,1]$5,

$t\in[0,1]$6

with $t\in[0,1]$7 the encoded clean three-frame video and $t\in[0,1]$8 (Tong et al., 15 Jan 2026).

4. CoF-Evol-Instruct dataset

To supervise the three-frame reasoning process, the framework introduces CoF-Evol-Instruct, a 64 K-sequence dataset of prompt-aligned “draft → refine → final” triples (Tong et al., 15 Jan 2026). The prompt pool contains 68 K unique prompts drawn from GenEval, Echo-4o, flow-GRPO, and self-generated prompts.

For each prompt, an anchor image is first sampled from one of three text-to-image models, selected from weak, medium, and strong tiers with probabilities 0.25, 0.5, and 0.25 respectively. A Qwen3-VL-8B judge then classifies each anchor as one of three states: $t\in[0,1]$9 (Semantically Misaligned), $\{z_1,z_2,z_3\}$0 (Visually Unrefined), or $\{z_1,z_2,z_3\}$1 (High-Fidelity) (Tong et al., 15 Jan 2026).

Trajectory construction is performed by a Unified Editing Primitive, implemented as a tripartite LLM-based agent consisting of planner, editor, and verifier. It applies minimal edits conditioned on five semantic categories: Attribute Binding, Object Combination, Quantity Control, Spatial Arrangement, and Context Manipulation. The anchor is expanded into $\{z_1,z_2,z_3\}$2 using one of three strategies: Forward Refinement ($\{z_1,z_2,z_3\}$3), Bidirectional Completion ($\{z_1,z_2,z_3\}$4), or Backward Synthesis ($\{z_1,z_2,z_3\}$5) (Tong et al., 15 Jan 2026).

The reported outcome is 64 K high-quality, defect-aware, three-frame trajectories evenly covering the five semantic categories. This dataset design makes the trajectory itself the object of supervision rather than merely the terminal image. A plausible implication is that the method attempts to encode a curriculum over defect correction, teaching the model not only what a successful output looks like but how an initially inadequate realization should be improved.

5. Training regime and empirical results

The model is fine-tuned from Wan2.1-T2V-14B for 1,800 steps on the 64 K CoF sequences, using batch size 64, AdamW, learning rate $\{z_1,z_2,z_3\}$6, and weight decay $\{z_1,z_2,z_3\}$7 (Tong et al., 15 Jan 2026). The VAE is frozen, independent frame encoding is used at $\{z_1,z_2,z_3\}$8 resolution, and inference generates a square three-frame chain at the same resolution, decoding only $\{z_1,z_2,z_3\}$9.

On GenEval, a benchmark for object-centric prompt following, the Wan2.1-T2V-14B base model attains 0.55, unified MLLMs such as BLIP3-o reach up to 0.84, and CoF-T2I achieves 0.86 (Tong et al., 15 Jan 2026). The reported gains span single-object, two-object, counting, colors, position, and attribute subtasks. On Imagine-Bench, which targets imaginative compositional prompts, Wan2.1-T2V base scores 5.939, the best unified MLLM baseline BAGEL-Think scores 6.930, and CoF-T2I reaches 7.468 (Tong et al., 15 Jan 2026).

The ablation study isolates two core design choices. A target-only fine-tune using only the final frame reaches GenEval 0.81, removing the independent VAE yields 0.83, and the full CoF-T2I system reaches 0.86 (Tong et al., 15 Jan 2026). Frame-wise performance further supports the progressive-refinement hypothesis: on GenEval, the successive frames score $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$0, $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$1, and $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$2 (Tong et al., 15 Jan 2026). These values indicate monotonic improvement across the trajectory rather than mere redundancy among frames.

Benchmark or setting	Result
GenEval, Wan2.1-T2V-14B base	0.55
GenEval, unified MLLMs up to	0.84
GenEval, CoF-T2I	0.86
Imagine-Bench, Wan2.1-T2V base	5.939
Imagine-Bench, BAGEL-Think	6.930
Imagine-Bench, CoF-T2I	7.468

The reported qualitative examples follow the same pattern. The draft frame retains coarse layout but may miss counts or colors; the refine frame corrects objects, relations, and attributes; the final frame adds textures, realistic shading, and backgrounds, including for contrived prompts such as “marble mule wearing a crystal saddle” (Tong et al., 15 Jan 2026).

6. Relation to visual reasoning and benchmarked limitations

CoF-T2I is explicitly framed as a pure-vision alternative to methods that rely on external verifiers or interleaved text planning (Tong et al., 15 Jan 2026). Its central claim is that visual reasoning can be internalized within a video-trained diffusion backbone if the model is supervised to produce interpretable refinement trajectories. This positions the method at the intersection of text-to-image generation, video generative modeling, and multimodal reasoning.

Its significance is sharpened by independent evidence that state-of-the-art text-to-image systems remain weak on visually grounded reasoning tasks requiring fine-grained state distinctions. The Commonsense-T2I benchmark evaluates whether models can distinguish adversarial prompt pairs that differ by a single commonsense-critical word, such as “a lightbulb without electricity” versus “a lightbulb with electricity,” and then render mutually exclusive expected outputs such as “the lightbulb is unlit” versus “the lightbulb is lit” (Fu et al., 2024). The benchmark consists of $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$3 pairwise examples, each with adversarial prompts, mutually exclusive expected output descriptions, a likelihood score $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$4 with only examples satisfying $z_{1:3}\sim p_\theta(Z_{1:3}\mid p).$5 retained, and a commonsense category from Physical Laws, Human Practices, Biological Laws, Daily Items, and Animal Behaviors (Fu et al., 2024).

On that benchmark, even DALL·E 3 with prompt revision achieves 48.92%, DALL·E 3 without revision 34.00%, and Stable Diffusion XL 24.92%, while open-source diffusion variants remain below 25% (Fu et al., 2024). The study further reports that GPT-4V (gpt-4o) aligned best with human judgments among automatic evaluators and that higher CLIP text-embedding similarity between adversarial prompts correlates with lower per-sample accuracy, reflecting prompt confusion (Fu et al., 2024).

This broader literature provides context for CoF-T2I’s design. A plausible implication is that explicit intermediate visual states may help counteract some failure modes of one-shot generation, especially when object counts, bindings, and spatial relations must be corrected progressively. However, no direct Commonsense-T2I result for CoF-T2I is provided in the source material, so any claim of superiority on commonsense-critical adversarial prompts would be unwarranted.

7. Interpretation, limitations, and significance

CoF-T2I demonstrates that a video model can be adapted into a text-to-image system without using motion as the target phenomenon. Instead, temporal structure is converted into an ordered latent reasoning scaffold. The method’s main architectural commitments are therefore interpretable intermediate frames, progressive visual refinement, and independent frame encoding to suppress motion artifacts (Tong et al., 15 Jan 2026).

A common misconception would be to treat CoF-T2I as simply a video-to-image reduction or a multi-frame ensemble. The reported training and ablation results indicate a more specific claim: the intermediate frames are intended to function as reasoning steps, and supervising the full trajectory yields better performance than supervising only the target frame (Tong et al., 15 Jan 2026). Another misconception would be to assume that the method relies on explicit textual decomposition; the paper instead emphasizes that CoF-T2I operates without any external verifier or textual planning, and that the reasoning remains in visual latent space (Tong et al., 15 Jan 2026).

The framework also raises methodological questions. Because the intermediate states are curated through CoF-Evol-Instruct and generated through a defect-aware editing pipeline, the refinement path is not wholly emergent but partly taught. This suggests that the observed Chain-of-Frame behavior is both an emergent capability of video models and an explicitly shaped training target. A further plausible implication is that future work may need to determine how much of the gain comes from temporal inductive bias, from trajectory supervision, and from the quality-control and editing procedures used to construct the dataset.

Within the current evidence, CoF-T2I establishes a distinct paradigm for text-to-image generation: a three-frame latent reasoning chain that uses a video backbone to perform progressive semantic and aesthetic correction, achieving 0.86 on GenEval and 7.468 on Imagine-Bench while exposing intermediate visual states as part of the generation process (Tong et al., 15 Jan 2026).