OmniGen2: An Open-Source Multimodal Generative Model

OmniGen2 is an open-source generative model designed to provide a unified framework for diverse multimodal generation tasks, including text-to-image synthesis, image editing, and in-context (subject-driven) generation. Distinctive architectural innovations, comprehensive training data pipelines, and a self-reflective generative mechanism position OmniGen2 as a reference point for the reproducible, instruction-following multimodal generation community (Wu et al., 23 Jun 2025 ).

1. Model Structure and Architectural Innovations

OmniGen2 introduces a bifurcated decoding architecture, explicitly separating the text and image generation pathways to maximize modality-specific capabilities and mitigate detrimental parameter sharing effects observed in prior multimodal generators.

• Text Decoding Pathway
  • Built atop a foundation multimodal LLM (MLLM), specifically Qwen2.5-VL-3B-Instruct, this branch manages all text understanding and production.
  • Operates in an autoregressive manner: $$\mathrm{P}(y|x) = \prod_t \mathrm{P}(y_t|y_{1:t-1}, x)$$, with $y$ as the text (or multimodal) output sequence and $x$ as the multimodal conditioning input.
  • Remains largely frozen throughout training, aside from minimal adaptation to accommodate a special image-generation token <|img|>; this preserves original LLMing performance.
• Image Decoding Pathway
  • Employs a transformer-based diffusion decoder for image generation, initialized independently of the text branch.
  • Incorporates a rectified flow-based diffusion process for improved generative performance.
  • Receives conditional hidden states from the frozen MLLM, as well as VQ-VAE encoded image features.
  • The VAE tokenizer is coupled exclusively to the image pathway; no VAE features contaminate or alter the MLLM.
• Distinct Parameterization
  • No parameters are shared between text and image branches, ensuring purity of modality-specific learning signals and avoiding the degradation of either branch’s competence.
• Positional Encoding (Omni-RoPE)
  • Defines a composite position space: $(id_{seq}, h, w)$, where $id_{seq}$ distinguishes modality or sequence, and $(h, w)$ provide local 2D spatial context within each image token sequence.
  • This design enables coherent spatial manipulation and supports multi-image tasks and fine-grained editing.

2. Data Construction Pipelines

OmniGen2 is trained on extensive and systematically constructed datasets targeting the full span of text-to-image, editing, and in-context generation challenges.

• Text-to-Image (T2I) Generation
  • Utilizes ~140 million open-source and 10 million proprietary images with synthetic annotations.
  • Ensures coverage across a wide array of subjects, styles, and compositional scenarios.
• Image Editing
  • Aggregates and curates sets such as SEED-Data-Edit, UltraEdit, OmniEdit, PromptFix, and ImgEdit.
  • Augments these with video-derived datasets, enabling the learning of complex edits such as temporal/action changes and multi-object movement.
  • Video-based samples are constructed by extracting bounding boxes, segmenting/tracking objects (GroundingDINO, SAM2), and performing outpainting or inpainting as needed.
• In-Context (Subject-Driven) Generation and Editing
  • Constructs instruction/context/target triplets from video sequences, facilitating learning of context-consistent subject-driven manipulations. Context images are sampled and outpainted based on tracked objects.
  • All editing and in-context data are described using high-quality, context-matched instructions generated by MLLMs, tying natural language descriptions directly to visual transformations.

3. Self-Reflective Generation: Multimodal Reflection Mechanism

OmniGen2 pioneers a reflection mechanism for image generation, extending the LLM "self-reflection" paradigm to the multimodal domain.

• Reflection Process
  • After initial image generation, the output is critiqued by an auxiliary MLLM (e.g., Doubao-1.5-pro), which identifies mismatches or deficiencies with respect to the instruction.
  • The model conditions a new generation on this critique, iterating—potentially in multiple reflection turns—to improve alignment and fidelity.
  • This mechanism targets common failure points such as omission of salient objects, attribute inaccuracies, or compositional errors, particularly in multi-object and instruction-heavy scenes.
• Reflection Dataset
  • Constructed instances follow the pattern: instruction → initial image → reflection critique → improved image.
  • Enables fine-tuning of the model’s multi-step reasoning and iterative self-correction abilities.

4. Evaluation and Benchmark Performance

OmniGen2 is extensively validated against mainstream multimodal benchmarks, often achieving state-of-the-art or highly competitive scores among open-source models.

• Text-to-Image (T2I) Benchmarks
  • GenEval: 0.86 with LLM rewriter (BAGEL: 0.88; UniWorld-V1: 0.84).
  • DPG-Bench: 83.57 (beating UniWorld-V1: 81.38; on par with SD3-medium: 84.08).
• Image Editing
  • Emu-Edit: Best CLIP-Out score (0.309), strong CLIP-I (0.876), DINO (0.822).
  • GEdit-Bench-EN: Near-top Semantic Consistency (SC) 7.16; overall 6.41.
  • ImgEdit-Bench: Sets open-source state-of-the-art, especially for action-based edits due to video training data.
• In-Context (Subject-Driven) Generation
  • OmniContext: Overall score 7.18, outperforming all open-source competitors (BAGEL: 5.73; UNO: 4.71).
  • Metrics: Prompt Following (PF), Subject Consistency (SC). Final score uses $$\mathrm{Overall} = \sqrt{\mathrm{PF} \times \mathrm{SC}}$$ where PF, SC are GPT-4.1-based ratings between 0–10.
• Qualitative Observations
  • Demonstrates robust multi-object compositionality, precise action/stylistic edits, and high contextual consistency.
  • No observed compromise of basic language understanding or text generation, attributable to the frozen MLLM branch.

5. Open-Source Release and Contributions

OmniGen2’s full open-source release encompasses models, training code, and all major datasets/pipelines, including:

• Pretrained MLLM, diffusion/image decoder, and reflection-augmented checkpoints.
• Scripts for the comprehensive data construction pipeline, facilitating reproducibility and further research.
• Release of the video-derived editing/in-context sets and reflection dataset, addressing coverage gaps in current open data resources.

This open release is intended to support and accelerate community-led research into instruction-following, multimodal alignment, and self-correcting generation.

6. Relation to OmniGenBench and Emerging Research Directions

OmniGen2 is both shaped by and contributes to the evolving ecosystem of multimodal benchmarks, particularly OmniGenBench (Wang et al., 24 May 2025 ).

• OmniGenBench provides a six-dimension taxonomy for systematic evaluation: appearance compliance, dynamics consistency, world knowledge anchoring, situational reasoning, spatial reasoning, and STEM-driven reasoning.
• The benchmark employs dual-mode evaluation—visual parsing for perception-centric tasks, and LLM-as-judge protocols (OmniScore) for cognition-centric tasks. OmniScore heavily weights instruction adherence.
• OmniGen2’s reflection mechanism and specialized data construction enable improvements precisely on the taxonomy’s most challenging tasks (e.g., symbolic/text-in-image, reasoning-heavy and multi-step scenarios).
• Continued development in the open-source space—including enhanced instruction tuning, better handling of deep STEM/professional knowledge, and unified benchmarks for human alignment verification—remain critical research frontiers.

Summary

OmniGen2 represents an integrated approach to advanced multimodal generation, combining architectural decoupling, curated data pipelines, and a self-correcting reflection mechanism. The model achieves competitive or leading open-source results in text-to-image, image editing, and in-context generation, validated on a suite of rigorous benchmarks. By releasing all model, code, and data artifacts, OmniGen2 establishes a foundation for reproducible research and continued innovation in instruction-following, context-consistent multimodal generative modeling.