Skywork UniPic: Unified 1.5B Model

• Skywork UniPic is a unified 1.5B autoregressive model that consolidates image understanding, text-to-image generation, and editing into a single end-to-end architecture.
• It employs dual visual encoders and a shared decoder to facilitate both pixel-level synthesis and semantic analysis, yielding high-fidelity, precise outputs.
• The model uses a progressive training strategy with reward-guided filtering on large-scale curated datasets, ensuring efficient performance on commodity hardware.

Skywork UniPic is a unified 1.5 billion-parameter autoregressive model designed for integrated image understanding, text-to-image generation, and image editing within a single architecture. It eliminates the reliance on task-specific adapters or inter-module connectors and demonstrates that comprehensive multimodal AI systems can be realized with state-of-the-art performance on commodity hardware. By leveraging a decoupled encoding strategy, progressive training, and large-scale curated datasets with task-specific reward models, Skywork UniPic achieves notable efficiency and deployability across core visual AI tasks.

1. Unified Autoregressive Architecture

Skywork UniPic is built around a unified autoregressive framework that consolidates multiple visual-linguistic tasks into one end-to-end pipeline. The architecture is characterized by two distinct visual encoders and a shared autoregressive decoder:

• Masked Autoregressive (MAR) Encoder–Decoder: Specialized for high-fidelity image synthesis, this component operates in a pixel-level autoregressive manner, dispensing with discrete codebooks to preserve photorealistic detail and enable fine-grained generation.
• SigLIP2 Encoder: Dedicated to visual understanding, this encoder extracts semantically-rich vision features via self-supervised losses and captioning-based pretraining. These high-level features are targeted at image understanding, retrieval, and compositional reasoning.
• Shared Embedding Space: Outputs from both encoders pass through two-layer MLP projection heads that map their representations into a common embedding space.
• Unified Decoder (Qwen2.5-1.5B-Instruct LLM Backbone): A 1.5B parameter autoregressive LLM serves as the shared decoder. This core module orchestrates instruction following, image synthesis, and editing by integrating pixel-level and semantic features.

This decoupling of visual encoding—while retaining a unified decoding process—enables efficient bidirectional knowledge transfer between image generation and conceptual understanding, thus avoiding the interoperability issues associated with modular multimodal systems.

2. Performance and Benchmarking

Skywork UniPic demonstrates competitive results across major multimodal and generation-specific benchmarks, indicating broad capability:

Metric	Value	Description
GenEval Score	0.86	Assesses instruction following and output-image/description alignment (strong compositionality)
DPG-Bench Record	85.5	Sets a new complex scene generation record (spatial relationships, multi-attribute prompts)
GEditBench-EN	5.83	Image editing evaluation: semantic consistency
ImgEdit-Bench	3.49	Image editing assessment: perceptual/semantic score

A GenEval score of 0.86 and DPG-Bench record of 85.5 place Skywork UniPic ahead of most prior unified models, while its GEditBench-EN and ImgEdit-Bench scores indicate that editing precision and fidelity are not compromised by unification. This suggests a robust balance between generation and editing capabilities at both pixel-level and semantic tiers.

3. Progressive Training Strategy

The training methodology for Skywork UniPic employs a progressive, resolution-aware schedule:

• Multi-Stage Curriculum: Model training begins at lower resolutions (e.g., 256×256), iteratively scaling to high resolutions (up to 1024×1024). This grants early-stage emphasis on basic visual learning with later-stage focus on details and photorealism.
• Dynamic Parameter Unfreezing: The model implements a staged unfreezing protocol encompassing pretraining, alignment, joint optimization, and supervised fine-tuning. For instance, at Stage 3, the LLM and visual encoders are jointly optimized under a multi-task loss:

$$L_{\text{total}} = \lambda_G L_{\text{Gen}} + \lambda_U L_{\text{Und}}$$

where $L_{\text{Gen}}$ is a diffusion/synthesis loss (e.g., $$L_{\text{Gen}} = \mathbb{E}_{\epsilon, t}[||\epsilon - \epsilon_\theta(x_t \mid t, z)||^2])$$ and $L_{\text{Und}}$ is a cross-entropy loss for understanding tasks.

• Capacity-Stability Tradeoff: The curriculum balances the network’s capacity for high-resolution detail synthesis with stability in alignment and semantic guidance across tasks.

A plausible implication is that such staged, resolution-aware, and dynamically-unfrozen curricula provide scalability without sacrificing convergence or multimodal alignment, especially at larger image sizes.

4. Dataset Construction and Task-specific Reward Models

• Curated 100 Million-Scale Datasets: The data curation process emphasizes balance between generation, editing, and understanding tasks, maximizing semantic diversity to prevent mode collapse and overfitting.
• Reward Models for Supervision: Two auxiliary models are leveraged:
  • Skywork-ImgReward: Trained (with Group Relative Policy Optimization, GRPO) to assess generation quality for filtering and weighting samples.
  • Skywork-EditReward: Similarly optimized for editing quality and semantic precision.
• Reward-Guided Filtering: Training samples scoring below a reward threshold (e.g., 0.9) are filtered out, aligning objectives more closely to perceived human quality and preference.

This strategy embeds externalized supervision to systematically enforce task-specific quality and semantic alignment, supporting efficient optimization of both unconditional and instruction-conditioned workflows.

5. Model Scale and Resource Efficiency

Skywork UniPic’s model size and resource requirements are optimized for practical deployment:

• Parameter Count: 1.5B parameters—approximately one-tenth the size of comparable systems such as BAGEL or UniWorld-V1—reduces computational and memory overhead without discounting performance.
• Hardware Compatibility: The architecture supports high-resolution image synthesis (1024×1024) on commodity hardware (e.g., RTX 4090, consuming <15 GB GPU memory).
• Efficiency Mechanisms: The decoupled encoder design, progressive curriculum, and resolution scaling mitigate overhead, enabling state-of-the-art results without prohibitively expensive infrastructure.

This efficiency broadens accessibility, supporting research and industrial scenarios where hardware scaling is a bottleneck.

6. Practical Task Spectrum

The unified architecture of Skywork UniPic directly supports a variety of real-world applications:

• Text-to-Image Generation: High-fidelity rendering of scenes specified by complex textual instructions, suitable for art, design, and creative prototyping.
• Image Editing: Robust competence in operations such as object addition/removal, style transfer, and attribute alteration, tailored to creative and commercial industries.
• Image Understanding: The SigLIP2 encoder provides semantic features critical for automated captioning, retrieval, and visual question answering.
• Interactive Multiturn Workflows: Users can fluidly alternate between detailed scene description, iterative image refinement, and targeted editing, leveraging bidirectional interaction between text and image modalities.

A plausible implication is enhanced user experience in creative, professional, or research-driven environments, facilitated by seamless multimodal interaction without task- or domain-specific adaptation.

7. Open Availability and Research Implications

All code, model checkpoints, and technical documentation for Skywork UniPic are openly hosted (e.g., Hugging Face), enabling reproducibility and further development. This open-source release is intended to accelerate research on deployable multimodal AI systems, offering a practical paradigm for unified, resource-efficient visual understanding and generation across domains.

By integrating task-specialized visual encoders with a shared LLM-based decoder, leveraging progressive training and reward-guided supervision on large-scale balanced data, Skywork UniPic demonstrates that high-quality multimodal AI can be effectively unified within a resource-conscious architecture. This modeling paradigm sets a new direction for efficient, general-purpose visual-linguistic models suitable for deployment beyond specialized research clusters.