EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture

Abstract: We propose EMMA, an efficient and unified architecture for multimodal understanding, generation and editing. Specifically, EMMA primarily consists of 1) An efficient autoencoder with a 32x compression ratio, which significantly reduces the number of tokens required for generation. This also ensures the training balance between understanding and generation tasks by applying the same compression ratio to images. 2) Channel-wise concatenation instead of token-wise concatenation among visual understanding and generation tokens, which further reduces the visual tokens in unified architectures. 3) A shared-and-decoupled network that enables mutual improvements across tasks while meeting the task-specific modeling requirements. 4) A mixture-of-experts mechanism adopted for visual understanding encoder, which substantially improves perceptual capabilities with a few parameters increase. Extensive experiments have shown that EMMA-4B can significantly outperform state-of-the-art unified multimodal approaches (e.g., BAGEL-7B) in both efficiency and performance, while also achieving competitive results compared to recent multimodal understanding and generation experts (e.g., Qwen3-VL and Qwen-Image). We believe that EMMA lays a solid foundation for the future development of unified multimodal architectures.

• The paper presents EMMA, a unified multimodal model that improves efficiency via high-compression visual encoding and channel-wise token concatenation.
• It leverages a shared-and-decoupled backbone with a mixture-of-experts router to balance semantic understanding and fine-grained generative detail.
• Experimental results show EMMA’s superior performance on multimodal benchmarks while using significantly fewer tokens compared to previous models.

EMMA: A Unified and Efficient Architecture for Multimodal Understanding, Generation, and Editing

Introduction and Motivation

Unified multimodal architectures have become pivotal for advancing research at the intersection of vision and language, with significant implications for both embodied AI and AGI. Despite the proliferation of unified frameworks, existing approaches are frequently bottlenecked by inefficient token utilization, imbalanced optimization across modalities, and suboptimal architectural unification. The paper "EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture" (2512.04810) addresses these deficiencies through a jointly optimized architecture—EMMA—designed for scalable multimodal understanding, generation, and editing capabilities.

Architectural Innovations

EMMA’s core advances are realized through system-level architectural decisions targeting both efficiency and expressivity:

• Efficient Visual Encoding: EMMA deploys a high-compression autoencoder, specifically DCAE, operating at a $32\times$ compression ratio, for both understanding and generation visual branches. This ensures cross-task balance in training and drastically lowers the number of processed visual tokens, significantly improving computational efficiency.
• Channel-wise Token Concatenation: In contrast to prior unified models (e.g., BAGEL) that concatenate visual tokens along the sequence dimension, EMMA achieves fusion via channel-wise concatenation. This design reduces context token counts without diminishing cross-modal feature interaction, enabling stronger performance with lower computation.
• Shared-and-Decoupled Backbone: The model backbone is configured to share shallower layers across tasks, leveraging mutual inductive bias, while deeper layers are decoupled to optimize for the idiosyncratic requirements of understanding (semantic modeling) and generation (semantic plus high-frequency detail modeling). This enables parameter sharing without negative cross-task interference.
• Mixture-of-Experts (MoE) Visual Encoder: Incorporating a router-augmented MoE into the SigLIP2-based visual encoder allows dynamic routing to a STEM-domain expert or a general expert submodule. This mitigates the inherent diversity in multimodal data (notably STEM images), enhancing overall perceptual capability with a minimal parameter increase.

  Figure 1: The EMMA architecture leverages adapters and a decoupled backbone to balance understanding and generation tasks within a single, unified multimodal model.

Data Construction Pipelines

EMMA’s results hinge on large, diverse, and carefully curated multimodal corpora for understanding, text-to-image (T2I) generation, and image editing:

• General Editing Pipeline: For image editing, instruction-image pairs are synthesized by leveraging vision-LLMs (VLMs) to generate and validate editing instructions, with downstream image editing models producing outputs. Further post-filtering, including face similarity for portraits, ensures instruction fidelity.

  Figure 2: The pipeline for constructing general image editing data tightly couples VLM-based instruction synthesis with model output validation.

• Text Editing Pipeline: For images featuring text, EMMA’s pipeline leverages OCR to localize text, synthesize plausible edit instructions (random replacement/removal), and validate edit completion, generating challenging text-edit benchmarks for both training and evaluation.

  Figure 3: The text editing data pipeline integrates text detection, instruction generation, edit synthesis, and rigorous OCR-based validation.

Experimental Results

EMMA-4B demonstrates a strong empirical profile across a comprehensive suite of multimodal benchmarks:

• Multimodal Understanding: EMMA achieves an MMVet score of 73.0 and MMBench score of 85.8, outperforming BAGEL-7B (67.2 MMVet, 85.0 MMBench) despite using significantly fewer parameters and one-fifth the visual context tokens.
• Text-to-Image Generation: EMMA delivers a GenEval score of 0.91 (0.93 with prompt rewriting), surpassing BAGEL-7B (0.82/0.88) and Qwen-Image (0.87), and attains 85.6 on DPG-Bench, rivaling models such as Qwen-Image.
• Image Editing: On GEdit-Bench-EN, EMMA achieves 6.53 overall, exceeding or matching prior unified models while using only 1/5 of the visual tokens compared to competitors.
• Generalization: Despite training predominantly with English instruction editing pairs, EMMA natively supports editing via instructions in Chinese and generates consistent results even under complex multi-step instructions.

  Figure 4: EMMA demonstrates high-fidelity text-to-image generation given complex and open-vocabulary textual prompts.

  

  Figure 5: EMMA’s editing results maintain subject consistency and instruction fidelity, even with multilingual or composite instructions—despite a training focus on single-English instructions.

Emergent Capabilities and Analysis

Two notable emergent phenomena are detailed:

1. Cross-Lingual and Complex Instruction Adherence: Without explicit Chinese editing or compositional instruction data, EMMA generalizes robustly to these domains. This is attributed to the architecture's end-to-end unification allowing robust transfer from understanding-heavy training data distributions.
2. Token Efficiency and Performance Tradeoff: EMMA’s reduced token count (enabled by high compression and channel concatenation) does not sacrifice performance. In image editing, EMMA uses one-fifth the token count of older approaches while attaining superior or equivalent evaluation scores, directly contradicting the widespread assumption that effective multimodal systems require extensive context tokens.

Theoretical and Practical Implications

EMMA establishes new precedents for efficiency, generalization, and modularity in unified multimodal models. The shared-and-decoupled architecture presents a compelling blueprint for optimization in scenarios with disparate task-specific characteristics, reminiscent of domain-adaptive transformer advances. The MoE design for the vision encoder confirms that methodologically lightweight expert routing suffices for robust generalization across visual domains, including STEM cases that typically degrade model-wide performance in large-scale VLMs.

Practically, the architecture allows deployment of competitively performant multimodal systems both in server and edge settings due to lower memory and computational footprints. The compositional training pipeline—bolstered by synthetic data generation and rigorous automated filtering—demonstrates scalable recipe for upcoming unified architectures addressing broader domains (e.g., video-text, multimodal RL).

Future Directions

EMMA signals several avenues for future research:

• Expanding MoE Expert Pools: The pipeline for dynamically routing visual tokens to downstream domain-specific experts beyond STEM can be generalized for continual learning and robust in-the-wild deployment.
• Improving Editing Consistency Metrics: The current image editing metrics exhibit limitations, particularly subject consistency. New objective criteria and benchmarks for region-based editing are warranted.
• Unified Training Paradigms: The cross-paradigm architecture, wherein next-token prediction and diffusion-based generative objectives co-exist, may inspire even broader joint training paradigms, unifying modality conditioning in a more principled fashion across both discrete and continuous domains.

Conclusion

EMMA delivers an integrated approach for large unified multimodal models, balancing token efficiency, empirical performance, and architectural flexibility. The innovations in visual token compression, channel-wise fusion, and mixture-of-experts routing, validated by consistent outperformance and cross-lingual generalization, make EMMA an instructive model for future work aiming to unify multimodal understanding, generation, and editing in a resource-efficient and extensible manner.