Large Language Models are Universal Reasoners for Visual Generation

Abstract: Text-to-image generation has advanced rapidly with diffusion models, progressing from CLIP and T5 conditioning to unified systems where a single LLM backbone handles both visual understanding and generation. Despite the architectural unification, these systems frequently fail to faithfully align complex prompts during synthesis, even though they remain highly accurate at verifying whether an image satisfies those same prompts. We formalize this as the \emph{understanding-generation gap} and propose UniReasoner, a framework that leverages the LLM as a universal reasoner to convert its understanding strength into direct generation guidance. Given a prompt, the LLM first produces a coarse visual draft composed of discrete vision tokens. It then performs a self-critique by evaluating the draft for prompt consistency, producing a grounded textual evaluation that pinpoints what needs to be corrected. Finally, a diffusion model is conditioned jointly on the prompt, the visual draft, and the evaluation, ensuring that generation is guided by explicit corrective signals. Each signal addresses a limitation of the other: the draft provides a concrete, scene-level anchor that reduces under-specification in text-only conditioning, while the evaluation turns verification into grounded, actionable constraints that correct omissions, hallucinations, and relational errors. Experiments show that UniReasoner improves compositional alignment and semantic faithfulness under the same diffusion backbone while maintaining image quality, demonstrating a practical way to exploit LLM reasoning to close the understanding-generation gap.

• The paper presents UniReasoner, a novel pipeline that leverages LLM self-critique to detect and correct prompt mismatches during image generation.
• It introduces a three-stage Draft–Evaluate–Diffuse approach that uses discrete vision tokens and explicit evaluative signals for improved compositional alignment.
• Empirical results show significant gains in constraint adherence, with improvements in counting (0.90 vs. 0.78) and spatial relations, outperforming baseline diffusion methods.

LLMs as Universal Reasoners for Visual Generation

Motivation and Problem Statement

Unified LLM frameworks for text-to-image synthesis, such as BAGEL and BLIP3-o, have shown significant improvements in architectural integration of visual understanding and generation. Despite utilizing the same LLM backbone for both tasks, these systems frequently fail to align generated images with complex prompt constraints: entity counts, spatial relations, and semantic attributes often diverge from specifications during synthesis, while the same model excels at evaluating those constraints post hoc. This understanding-generation gap persists across architectures, exposing a fundamental asymmetry in the generative capabilities versus evaluative strength of large-scale LLMs.

Figure 1: The understanding-generation gap revealed by unified LLM models, where the model fails to satisfy explicit prompt constraints during generation but demonstrates precise evaluative accuracy.

UniReasoner Framework: Draft–Evaluate–Diffuse

To leverage the rigorous evaluative strength of LLMs as actionable guidance for visual generation, the paper proposes UniReasoner—a three-stage pipeline that explicitly transforms multi-modal reasoning into corrective signals for diffusion-based synthesis. This framework operates as follows:

1. Draft Stage: The LLM generates a visual draft $d$ in the form of discrete vision tokens using SigLIP-2 quantization, providing a semantically grounded spatial plan compatible with autoregressive text token formats and the LLM’s internal world model.
2. Evaluate Stage: The same LLM performs a self-critique of the draft, producing a grounded evaluation $e$ that identifies prompt-draft mismatches, yielding explicit “what-to-fix” diagnostic signals.
3. Diffuse Stage: A diffusion model conditions its generation on the tuple $(p, d, e)$, utilizing the draft and evaluative signals for spatial anchoring and targeted correction during denoising.

   Figure 2: UniReasoner pipeline—LLM as universal reasoner via Draft, Evaluate, and Diffuse stages, providing explicit corrective signals to the diffusion backbone for high-fidelity synthesis.

Implementation and Training Protocol

The LLM backbone is instantiated with Qwen; the diffusion generator employs SANA. All diffusion parameters are frozen to isolate the reasoning contribution. The draft stage discretizes SigLIP-2 features via vector quantization, ensuring the visual tokens encode high-level semantic primitives, rather than low-level pixel artifacts, thereby aligning both drafting and critique with the LLM’s world knowledge. Grounded evaluation relies exclusively on LLM self-critique, operationalized as a diagnosis of prompt-draft discrepancies with explicit verbalization (e.g., missing objects, incorrect spatial relations).

Training utilizes two phases: image reconstruction with synthetic drafts and evaluations, followed by hard-negative finetuning using challenging prompt-draft pairs scored by Qwen-VL. These datasets are encoded as quadruples $(p, d, e, I^t)$. Optimization targets both the LLM and cross-modal connectors, with image diffusion generator weights frozen.

Empirical Results: Compositional Alignment and Faithfulness

UniReasoner surpasses all baselines in compositional alignment under GenEval ($0.88$ vs. SANA’s $0.79$), with category-wise improvements on counting ($0.90$ vs. $0.78$), position ($0.83$ vs. $0.62$), and attribute binding ($e$0 vs. $e$1). Similar gains manifest on DPG-Bench ($e$2 vs. $e$3 overall, $e$4 vs. $e$5 on global instructions), despite the underlying diffusion generator being identical across compared baselines. This demonstrates that explicit reasoning via draft and evaluation stages, rather than backbone scaling, yields improved prompt adherence.

Ablation studies show that visual draft tokens derived from SigLIP-2 feature quantization achieve superior compositional constraint satisfaction compared to VAE latents or generic VQ token drafts. Notably, multi-source conditioning with draft and evaluation signals enables highly localized semantic corrections, distinct from text-only prompt rewriting approaches.

Figure 3: The full reasoning process of UniReasoner, illustrating draft generation, prompt-draft evaluation, and the targeted corrections made during diffusion synthesis.

Figure 4: Qualitative text-to-image samples generated by UniReasoner across photorealistic and artistic prompts, exhibiting improved compositional faithfulness.

Practical and Theoretical Implications

UniReasoner's explicit integration of LLM reasoning into visual synthesis pipelines provides a principled mechanism to bridge the gap between high-level specification and generative fidelity. The pipeline demonstrates that the evaluative and diagnostic capacities of LLMs can be operationalized as corrective signals without architectural modification of diffusion models. Practically, this enables the design of modular, reasoning-enhanced systems for controllable, constraint-aware text-to-image generation. Theoretically, the approach supports a broader paradigm for generative modeling, in which reasoning and understanding are not merely upstream text encoders but inform every stage of visual synthesis through grounded, actionable guidance.

This model opens directions for future research into universal reasoning interfaces for multi-modal generation, reward-based correction of drafts and evaluations, and the extension of similar reasoning-enhanced mechanisms to other generative modalities (video, 3D, audio). The use of discrete, semantically aligned vision tokens for draft generation may influence future work on efficient, autoregressive multi-modal sampling.

Conclusion

The paper introduces UniReasoner, employing LLMs as universal reasoners to bridge the understanding-generation gap in text-to-image diffusion models. By explicitly orchestrating draft construction, self-critique, and corrective guidance during synthesis, UniReasoner achieves superior semantic faithfulness and compositional alignment compared to prior unified models, without modifications to the diffusion backbone. This approach demonstrates that leveraging the evaluative strengths of LLMs as multi-modal controllers is vital for constraint-compliant visual generation and signals future research into reasoning-driven generative architectures.