Understanding-in-Generation (UiG) Insights

• Understanding-in-Generation (UiG) is defined as a framework where a model actively leverages its own semantic understanding to regulate and improve its generative outputs.
• UiG employs mechanisms like intrinsic self-rewarding, gradient-based supervision, iterative editing, and chain-of-thought verification to enhance generation quality.
• Empirical studies show that integrating understanding as a supervisory signal in unified multimodal models yields substantial gains in generation performance while partly addressing the asymmetry between understanding and generation.

Understanding-in-Generation (UiG) denotes a research program in unified multimodal models (UMMs) in which understanding is treated as an active source of control, supervision, or feedback for generation rather than as a merely coexisting capability. In the narrower sense made explicit by "Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generation" (Lyu et al., 23 Sep 2025) and "Learning to Generate via Understanding" (Pan et al., 6 Mar 2026), UiG means that a model inspects its own generations, judges them through its understanding pathway, and uses that judgment to improve future outputs. In the broader sense adopted by later benchmark and transfer studies, UiG also includes understanding-guided generation, generation-guided understanding, and iterative mutual enhancement inside a single multimodal loop (Liu et al., 25 Jun 2026).

1. Historical emergence and the understanding-generation gap

UiG emerged from a recurrent diagnosis in UMMs: understanding and generation are nominally unified, but their capabilities are asymmetric and often poorly coupled. A central claim in "Learning to Generate via Understanding" is that UMMs usually exhibit a stronger visual understanding branch than generation branch; the model can often recognize fine-grained semantic details, spatial relations, counts, colors, and attributes, yet fails to faithfully realize those details in text-to-image generation (Pan et al., 6 Mar 2026). The paper further argues that current systems train understanding and generation largely separately, which creates a mismatch and can even induce negative transfer.

The same diagnosis appears in more general evaluative work. "Quantifying the Gap between Understanding and Generation within Unified Multimodal Models" argues that current models often achieve engineering-level unification rather than deep cognitive convergence: the same knowledge may be accessible in one modality but not reliably expressed in the other, and capability emergence and knowledge across modalities are unsynchronized (Wang et al., 2 Feb 2026). "Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward" reaches a related conclusion from controlled synthetic tasks, describing a pronounced understanding-generation gap in both reasoning generation and knowledge transfer (Niu et al., 25 Nov 2025).

This diagnosis motivated a shift from viewing asymmetry as a defect to viewing it as a training signal. In this framing, UiG does not assume that joint training alone will produce synergy. Instead, it asks how the stronger side of a unified model can directly regulate the weaker side, or how generative processes can in turn sharpen understanding.

2. Core mechanisms for turning understanding into generation

The most explicit UiG mechanism is the understanding-driven intrinsic reward introduced as GvU. Given a prompt $T=T_{1:L}$ and a generated image $I$, the understanding branch evaluates how likely the prompt tokens are under that image, with intrinsic reward defined as the geometric mean of token probabilities: $$P(T_{1:L}|I)= \left(\prod_{j=1}^L p_\theta(T_j|X_{j-1})\right)^{1/L}.$$ Here $X_0=[I]$, each token probability is produced autoregressively, and the token is appended back into context after scoring. The resulting reward is dense and token-level rather than image-level. The same work uses self-supervised reinforcement learning with GRPO: the generation branch acts as student, the understanding branch as teacher, and training proceeds by generating a group of candidate images, scoring them intrinsically, normalizing rewards into advantages, and updating the policy toward outputs that better satisfy the prompt (Pan et al., 6 Mar 2026).

A second mechanism is direct gradient routing from understanding into generation. "Steering Visual Generation in Unified Multimodal Models with Understanding Supervision" introduces Understanding-Oriented Post-Training (UNO), which keeps the understanding expert frozen and feeds noised generative representations into it. The understanding expert then performs two proxy tasks: captioning for semantic abstraction and visual regression for structural details. These losses are added to the standard generative loss,

$\mathcal L_{\mathrm{total} = \mathcal L_{\mathrm{mse} + \lambda_1 \mathcal L_{\mathrm{language} + \lambda_2 \mathcal L_{\mathrm{vision},$

with $\lambda_1 = 0.1$ and $\lambda_2 = 0.2$, so that gradients from understanding objectives reshape the generative latent space without changing the architecture (Liu et al., 7 May 2026).

A third mechanism uses iterative diagnosis and repair. In the explicit UiG framework of (Lyu et al., 23 Sep 2025), the model first generates an initial image, then verifies the image with an understanding prompt, produces an editing instruction if the image does not match the text, edits the image, and repeats until the understanding module returns a match. The defining claim is that image editing serves as the bridge that converts understanding into actionable generation guidance.

A fourth mechanism appears at inference time. "UniGen" introduces Chain-of-Thought Verification (CoT-V), in which the model generates multiple candidate images, decomposes the prompt into atomic visual questions, answers them step by step for each image, and selects the best candidates by verification score. In that setting, understanding operates as a self-verifier for generation quality rather than as a training-time loss alone (Tian et al., 20 May 2025).

Taken together, these methods suggest that UiG is not a single algorithmic recipe. It includes intrinsic self-rewarding, frozen-teacher supervision, iterative editing, and test-time self-verification, all organized around the same principle: generation should be regulated by semantic judgments internal to the unified model.

3. Conceptual taxonomy and evaluation protocols

A broad taxonomy of UiG is provided by Unison, which evaluates UMMs along four dimensions: Internal Consistency, Understanding-Guided Generation (UGG), Generation-Guided Understanding (GGU), and Mutual Enhancement (ME). Internal consistency measures whether understanding and generation agree semantically and explicitly penalizes spurious consistency, where a model misunderstands an image and then generates an image consistent with its own error. UGG measures whether a model can use its own reasoning to guide generation, for example by localizing an edit region before editing. GGU measures whether intermediate generation improves reasoning. ME evaluates iterative self-refinement loops that alternate between generation, judgment, refinement instruction, and regeneration (Liu et al., 25 Jun 2026).

IMUG-Bench extends the evaluation problem from isolated turns to multi-turn interleaved image-text dialogue. It comprises three classes—Static Spatial, Temporal Causal, and Hybrid—over 3,113 samples and 12,034 interaction turns, with 2 to 6 turns per sample and an average 3.87 turns per task. The benchmark mixes Static-MCQ, Dynamic-MCQ, and Image-Generation turns, so a model must understand fixed context, understand its own prior generations, and preserve cross-turn consistency when editing or generating images. Generation is judged by a VLM-as-a-Judge protocol over evaluation-points that separately score image update requirements and image consistency checks (Meng et al., 8 Jun 2026).

GapEval approaches UiG as bidirectional coherence. Each item can be answered in both modalities, allowing a symmetric evaluation of understanding and generation on the same semantic content. Outputs are judged for correctness, and the gap is then estimated with Multidimensional Item Response Theory to separate latent ability from item difficulty. The resulting gap score is intended to quantify whether knowledge is genuinely shared across modalities or remains disjoint (Wang et al., 2 Feb 2026).

UniSandbox uses controlled synthetic, out-of-distribution tasks to isolate two specific UiG bottlenecks: reasoning generation and knowledge transfer. Its procedure captions generated images with an MLLM and compares those captions against ground truth through a second semantic judge, so the measured signal concerns semantic consistency rather than aesthetics. This design is meant to avoid leakage and confounding memorization effects (Niu et al., 25 Nov 2025).

These benchmarks collectively formalize a central UiG distinction: success on understanding tasks and success on generation tasks do not, by themselves, establish synergy. The more stringent question is whether one modality reliably improves, verifies, or reconstructs the other.

4. Empirical findings and architectural conditions

The strongest direct evidence for understanding-to-generation transfer comes from methods that make the understanding branch an internal teacher. In GvU, training on only 50,000 text prompts with no external image dataset for the RL phase improves X-Omni on GenEval from 0.68 to 0.81, and to 0.84 with the $\dagger$ rewriting variant, corresponding to a 19.1% relative increase. The same method reaches 85.68 on DPG-Bench and improves GenEval++ from 0.282 to 0.404, a 43.3% gain. On the understanding side, MMT-Bench improves from 49.76 to 49.92, with gains concentrated in fine-grained subtasks such as Visual Illusion, Hallucination, VCR, and DKR (Pan et al., 6 Mar 2026).

UNO reports a similar but gradient-based pattern on BAGEL-7B. Adding understanding supervision improves GenEval2 from 71.7 to 75.1, DPG-Bench from 84.03 to 86.12, and UniGenBench++ from 61.53 to 65.03. For editing, GEdit-Bench-EN improves from 6.52 to 7.17 and GEdit-Bench-CN from 6.50 to 7.20. Ablations show that language supervision alone and visual supervision alone both help over SFT, while joint supervision works best, supporting the claim that semantic abstraction and structural detail are complementary (Liu et al., 7 May 2026).

Transferability studies make the architectural dependency explicit. "Transferability Between Understanding and Generation in Unified Multimodal Models" reports that cross-task capability transfer is strongest in models with a fully shared transformer backbone and a unified visual encoder, weaker in partially coupled designs, and weak or absent in more decoupled systems. On counting, understanding training improves Lumina-DiMOO’s generation accuracy from 48.0% to 57.0% and reduces MAD from 1.11 to 0.83. On spatial relation, understanding training raises generation accuracy from 67.0% to 74.0%, whereas direct generation training reaches 80.0% but induces more distribution shift and less natural image quality. Text recognition/generation transfer exists but is weaker, which the paper attributes to the need for fine-grained pixel-level detail (Kang et al., 5 Jul 2026).

A recurring empirical pattern is therefore twofold. First, UiG effects are measurable and often substantial on generation. Second, they are contingent: stronger transfer appears when representations are genuinely shared, when the understanding branch is already competent, and when the target capability is structurally aligned with what understanding supervision can encode.

5. Domain extensions beyond standard image generation

UiG has been generalized beyond text-to-image generation into domains where the relation between understanding and generation is either structurally tighter or more operationally constrained. In video, "Uni-ViGU" inverts the usual design and begins from a pretrained text-to-video diffusion transformer, WAN2.1. Its unified flow combines continuous flow matching for video and discrete flow matching for text, while Knowledge Recall and Capability Refinement transfer generation knowledge into understanding. The paper describes the result as competitive performance on both video generation and understanding, supporting a generation-centric route to unification (Qin et al., 9 Apr 2026).

In 3D, two complementary formulations appear. "Omni-View" argues that generation facilitates understanding in multiview scenes: the model jointly learns scene understanding, novel view synthesis, and geometry estimation, and achieves 55.4 on VSI-Bench while also reporting strong performance in novel view synthesis and 3D scene generation (Hu et al., 10 Nov 2025). "UniUGG" frames 3D UiG around a geometric-semantic encoder, a spatial decoder, and latent diffusion for target-view representation generation, and is described as outperforming the second best on VSI-Bench by 17.9% while supporting both spatial VQA and 3D generation (Xu et al., 16 Aug 2025).

In medicine, SynerMedGen argues that only generation-aligned understanding benefits synthesis. It constructs three understanding tasks—CTS, MI, and TIA—from the same paired synthesis corpus used for downstream generation, uses a two-stage GAU/UCG training strategy, and reports strong zero-shot performance across 22 medical image synthesis tasks from understanding training alone. The released SynerMed dataset contains 1M paired synthesis samples and 2M generation-derived understanding instances (Zhao et al., 9 May 2026).

In time series, TimeOmni-VL converts sequences into images through Bi-TSI, performs structured understanding over TS-images, and then uses a calibrated Chain-of-Thought as a control signal for forecasting and imputation. Its ablation shows that removing the understanding model and disabling CoT at inference causes an average 8.2% increase in nMASE, indicating that reasoning is materially involved in generation rather than merely explanatory (Guan et al., 19 Feb 2026).

Domain-specific applications exhibit the same logic in more specialized forms. UI-UG unifies UI understanding and UI generation through a shared MLLM, GRPO for understanding, DPO for generation, and a JSON-based DSL workflow; the paper reports that integrating understanding and generation improves both tasks and that DPO raises the MLLM-based generation score from 36.7 to 42.02, about 14.5% (Yang et al., 29 Sep 2025). UniCTokens uses a single set of unified concept tokens across personalized understanding, concept generation, and personalized knowledge-driven generation, reporting a knowledge-driven generation score of 0.359 versus 0.266 for Yo’Chameleon on UnifyBench (An et al., 20 May 2025). Scone uses the understanding expert as a semantic bridge for subject distinction, composition, and masking in subject-driven image generation, and reports the highest composition, distinction, and overall scores among open-source models on SconeEval (Wang et al., 14 Dec 2025).

6. Misconceptions, limitations, and open directions

A frequent misconception is that a model capable of both understanding and generation is therefore already unified in a meaningful UiG sense. Multiple studies reject this. Unison finds that strong single-task performance does not imply strong unified performance and that internal consistency is fundamental to broader unified competence (Liu et al., 25 Jun 2026). IMUG-Bench further shows that current UMMs remain weak in multi-turn interleaved settings, that Dynamic-MCQ is harder than Static-MCQ, and that image-generation performance declines as turns increase because early mistakes contaminate later context through exposure bias (Meng et al., 8 Jun 2026).

Another misconception is that improved generation necessarily feeds back strongly into understanding. The evidence is more limited. GvU reports clear secondary gains in fine-grained understanding, but also states that the understanding gains are relatively small compared with generation gains and that the method does not fully close the multimodal capability gap (Pan et al., 6 Mar 2026). This suggests that UiG can narrow the asymmetry without eliminating it.

A deeper limitation concerns representation sharing itself. GapEval argues that knowledge updates are often modality-local and that present-day UMMs are best characterized as surface-unified systems with incomplete cognitive coherence (Wang et al., 2 Feb 2026). UniSandbox reports that explicit Chain-of-Thought can bridge reasoning generation and aid knowledge retrieval, but also shows that knowledge transfer remains brittle, inverse retrieval remains difficult, and self-training depends on high-quality CoT data, rejection filtering, and curriculum learning (Niu et al., 25 Nov 2025).

Benchmarking also remains imperfect. IMUG-Bench notes that evaluation still relies on judge models and structured scoring rather than pure human annotation for every sample, while history-format differences complicate exact comparability across model families (Meng et al., 8 Jun 2026). Unison addresses part of this by introducing Unison-Judge, a dedicated evaluator aligned with human judgments, but its own results still highlight how far current models remain from robust synergy (Liu et al., 25 Jun 2026).

The surveyed literature therefore points toward a consistent design implication. A plausible implication is that future UMMs will require stronger internal self-evaluation mechanisms, more deliberate representation sharing, and training curricula that force understanding and generation to interact at the level of the same capability rather than merely coexist within the same architecture. In that sense, UiG is less a single method than a criterion for multimodal unification: understanding should actively regulate generation, generation should become available as a reasoning aid for understanding when appropriate, and both directions should remain semantically consistent over time and across tasks.