UniPic2-SD3.5M-Kontext: Unified Gen & Edit

• The paper introduces a unified image model that leverages architectural modifications, including the injection of reference image latents, to jointly support text-to-image generation and image editing.
• It employs mixed-task pre-training with high-quality data and Progressive Dual-Task Reinforcement to achieve superior performance metrics (e.g., GenEval 0.89, GEdit-En 6.59) compared to larger alternatives.
• The model integrates within the Skywork UniPic 2.0 framework, connecting with Qwen2.5-VL-7B to deliver robust multimodal understanding, generation, and editing in a single system.

UniPic2-SD3.5M-Kontext is a 2B-parameter diffusion transformer (DiT) image model retrofitted from SD3.5-Medium to support both text-to-image generation and image editing within a single unified model. In the Skywork UniPic 2.0 framework, it functions as the generation-and-editing component that is later connected to Qwen2.5-VL-7B to form UniPic2-MetaQuery, a unified multimodal system spanning understanding, generation, and editing. The central claim associated with the model is that careful architectural modification, large-scale mixed pre-training, and staged online reinforcement can outperform substantially larger open-source alternatives, including BAGEL and FLUX.1-Kontext, without relying primarily on parameter scaling (Wei et al., 4 Sep 2025).

1. Model definition and lineage

UniPic2-SD3.5M-Kontext begins from SD3.5-Medium and is adapted in two major ways. First, it is retrained on a mixed corpus of generation and editing data rather than only text-to-image data. Second, reference image latents are injected into the DiT self-attention layers so that the model conditions jointly on a text instruction and a reference image. This converts the original text-only generator into a joint generation-and-editing model (Wei et al., 4 Sep 2025).

The model is positioned as a compact alternative to much larger unified systems. The paper explicitly frames it against BAGEL and FLUX.1-Kontext, arguing that training strategy and architectural design can be more consequential than brute-force scaling alone. A plausible implication is that UniPic2-SD3.5M-Kontext is intended not merely as an editing extension of SD3.5-Medium, but as a test case for a broader claim about modular multimodal system design.

Within Skywork UniPic 2.0, UniPic2-SD3.5M-Kontext is not the terminal system. Rather, it is the generation-and-editing backbone that later becomes part of UniPic2-MetaQuery via a connector to Qwen2.5-VL-7B. Its role is therefore dual: it is both a standalone unified image model and a component in a larger multimodal architecture.

2. Architectural modifications and conditioning mechanism

The principal architectural change is the injection of VAE latents from the reference image into the DiT’s self-attention layers. The text encoder converts the instruction into embeddings, the VAE encodes the reference image into latents, those latents are projected into context tokens, and the context tokens are concatenated with the target image’s noise tokens. The model then processes a single combined sequence, with positional encoding distinguishing reference-image tokens from target-image tokens (Wei et al., 4 Sep 2025).

This conditioning scheme is designed to preserve structure and texture in editing. Because the model receives both instruction tokens and reference-image-derived tokens inside the same attention stack, it can jointly represent semantic transformation targets and source-image constraints. The paper associates this with better preservation of source structure and more consistent edits.

Training alternates image-generation batches and image-editing batches so that both tasks are jointly optimized. This is a straightforward but consequential design choice: generation and editing are not separated into distinct models or isolated phases during pre-training. Instead, the mixed-task regime is present before reinforcement learning.

The model also uses progressive resolution training and balanced sampling across the common aspect ratios 1:1, 4:3, 3:2, and 16:9. The stated effect is native dynamic-resolution generation at both train and inference time, together with improved generalization to varied layouts. This suggests that the architecture is configured not only for unified task support but also for native variation in spatial format.

3. Pre-training data and optimization setup

The paper distinguishes between the alignment stage used for the later connector-based multimodal system and the direct pre-training of UniPic2-SD3.5M-Kontext itself (Wei et al., 4 Sep 2025).

For connector and alignment pre-training, the setup uses 150M image-text pairs for 500K steps with global batch size 1,024 and learning rate $1\mathrm{e}{-4}$. The data consists of 30M open-source samples from CC12M, Megalith10M, RedCaps, and LAION-Aesthetics, together with 120M internal synthetic images.

For UniPic2-SD3.5M-Kontext, the actual Kontext model is trained on 5M image editing samples and 6M text-to-image samples for 200K steps with global batch size 128 and learning rate $4\mathrm{e}{-5}$. The editing data sources include UniWorld-V1, OmniGen2, NHR-Edit, ShareGPT4o-Image, GPT-Image-Edit-1.5M, and 450K internal samples. The text-to-image corpus includes 5.6M internal high-quality real images and 400K public samples from BLIP3-o and ShareGPT4o-Image.

The paper emphasizes that these are high-quality datasets and that substantial internal data contributes to fidelity and robustness. That emphasis is important because the paper’s broader thesis is that quality-focused training strategy can be more effective than merely increasing parameter count. A plausible implication is that the reported performance should be interpreted as a joint consequence of architecture, optimization schedule, and curation strategy, rather than architecture alone.

4. Progressive Dual-Task Reinforcement

A central contribution is Progressive Dual-Task Reinforcement (PDTR), a staged reinforcement learning strategy designed for unified text-to-image and image-editing optimization. The motivation is that the two tasks differ in input modality, output distribution, and evaluation criteria. The paper argues that naive joint optimization can cause gradient conflict, performance degradation, and classic multitask negative transfer (Wei et al., 4 Sep 2025).

PDTR addresses this by decoupling the optimization order. The schedule first reinforces image editing and then reinforces text-to-image generation. The stated intent is to strengthen one task and then improve the other without destroying the first.

The reinforcement algorithm is Group Relative Policy Optimization (GRPO), following Flow-GRPO. For a text hidden state $h$, the model samples a group of $G$ images $\{x_0^i\}_{i=1}^{G}$ and trajectories $\{x_T^i, x_{T-1}^i, \ldots, x_0^i\}_{i=1}^{G}$. The normalized advantage is defined as

$$A_i = \frac{R\left(x_0^i, h\right)-\operatorname{mean}\left(\left\{R\left(x_0^i, h\right)\right\}_{i=1}^{G}\right)}{\operatorname{std}\left(\left\{R\left(x_0^i, h\right)\right\}_{i=1}^{G}\right)}.$$

The objective uses a clipped policy-ratio term and a KL penalty, with $r_t^i(\theta)$ defined as the ratio of current to old transition probabilities. The symbols identified in the paper are $\theta$ for current policy parameters, $\mathcal{D}$ for the training data distribution, $4\mathrm{e}{-5}$0 for the current policy, $4\mathrm{e}{-5}$1 for the reference policy, $4\mathrm{e}{-5}$2 for the number of diffusion steps, $4\mathrm{e}{-5}$3 for the group size, $4\mathrm{e}{-5}$4 for the clipping threshold, $4\mathrm{e}{-5}$5 for the KL penalty weight, and $4\mathrm{e}{-5}$6 for KL divergence.

In Stage 1, image-editing reinforcement optimizes for semantically faithful, visually natural, and structurally consistent edits. The reward sources are Skywork-EditReward, described as a self-trained reward model, and online GPT-4.1 evaluation. The reported effects are improved precise edits, consistency, and preservation of source structure.

In Stage 2, text-to-image reinforcement uses GenEval as a verifiable reward together with classical detectors and layout analysis tools such as MMDetection and Mask2Former-style analysis to assess compositional accuracy and instruction following. The reported effects are improved semantic structure, instruction adherence, and fine-grained compositional generation.

The key empirical claim is that the stages are mutually beneficial rather than mutually destructive: editing RL helps text-to-image, text-to-image RL helps editing, and reinforcing text-to-image after editing does not degrade editing performance.

5. Ablations and benchmark performance

The ablation study is presented as direct evidence for the absence of negative interference. The baseline without GRPO scores GenEval $4\mathrm{e}{-5}$7, DPG $4\mathrm{e}{-5}$8, GEdit-EN $4\mathrm{e}{-5}$9, and ImgEdit $h$0 (Wei et al., 4 Sep 2025).

Editing-only reinforcement improves editing while leaving generation intact. With GPT-4.1, GenEval remains $h$1, DPG improves to $h$2, GEdit-EN rises to $h$3, and ImgEdit rises to $h$4. With EditReward, GEdit-EN reaches $h$5 and ImgEdit $h$6. Generation-only reinforcement with GenEval raises GenEval from $h$7 to $h$8, improves DPG modestly, and leaves editing stable. Under joint PDTR, GPT-4.1 plus GenEval yields GenEval $h$9, DPG $G$0, GEdit-EN $G$1, and ImgEdit $G$2, while EditReward plus GenEval yields GenEval $G$3, DPG $G$4, GEdit-EN $G$5, and ImgEdit $G$6. The paper interprets these results as positive cross-task transfer and synergistic reinforcement.

For direct model comparison, the reported generation results are as follows:

Model	GenEval	DPG
UniPic2-SD3.5M-Kontext	0.89	84.23
FLUX.1-dev	0.67	84.00
SD3.5-Medium	0.65	83.86
BAGEL	0.88	85.07
Blip3-o-8B	0.84	81.60
MetaQuery-XL	0.80	82.05

The paper emphasizes that UniPic2-SD3.5M-Kontext beats BAGEL on GenEval, is competitive on DPG, and outperforms much larger unified systems in instruction following.

The reported editing results are:

Model	GEdit-En	ImgEdit
UniPic2-SD3.5M-Kontext	6.59	4.00
FLUX.1-Kontext	6.26	3.52
BAGEL	6.52	3.20
Step1X-Edit	6.97	3.06

These comparisons are used to support the claim that a 2B-parameter generation backbone can exceed or match much larger editing-capable systems. The paper also mentions qualitative comparisons showing better object counts, better positions, better colors, and more faithful style adherence.

A common misconception in this context is that unified generation-and-editing performance is determined primarily by model size. The reported ablations and cross-model comparisons are presented specifically to contest that assumption.

6. Integration into UniPic2-MetaQuery and the Skywork UniPic 2.0 framework

UniPic2-SD3.5M-Kontext is connected to Qwen2.5-VL-7B through a learnable bridge to form UniPic2-MetaQuery. The overall architecture comprises a frozen MLLM, Qwen2.5-VL-7B; a diffusion generator, initially SD3.5-Medium and later replaced by UniPic2-SD3.5M-Kontext; a learnable query mechanism; and a transformer-based connector with 24 layers and about 1B parameters (Wei et al., 4 Sep 2025).

Training proceeds in two stages. In the alignment stage, both Qwen2.5-VL and SD3.5-Medium are frozen while the connector and learnable queries are trained on large-scale text-to-image data. In the unified training stage, SD3.5-Medium is replaced by UniPic2-SD3.5M-Kontext, the connector is unfrozen, and the connector together with the Kontext model is jointly fine-tuned on high-quality generation and editing data.

This joint training enables the final system to support understanding, generation, and editing. For editing, the system feeds both text instructions and reference images into the MLLM, while VAE latents go to the DiT. The paper characterizes this as semantically rich conditioning.

An important ablation result is that the best UniPic2-MetaQuery performance is obtained when both the connector and the DiT are unfrozen. The reported interpretation is that joint adaptation benefits both the bridge and the generator.

The understanding branch remains frozen, preserving Qwen2.5-VL’s strengths. The paper reports qualitative evidence for universal recognition, multi-instance recognition, scene understanding, object grounding with structured JSON output, and OCR, and states that the model achieves strong quantitative results on MMBench, MMMU, and MM-Vet.

Skywork UniPic 2.0 is the umbrella framework spanning UniPic2-SD3.5M-Kontext and UniPic2-MetaQuery. The framework is described as a simple, scalable, modular route to unified multimodal modeling: use strong pretrained components, connect them with a learnable bridge, apply large-scale pretraining, and then use task-aware RL with PDTR. The paper’s key thesis is that careful training and modular integration can outperform just scaling model size. This suggests a research direction in which multimodal system quality is pursued through component specialization and optimization strategy rather than monolithic enlargement alone.