TUNA: Taming Unified Visual Representations for Native Unified Multimodal Models (2512.02014v1)

Abstract: Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present TUNA, a native UMM that builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder. This unified representation space allows end-to-end processing of images and videos for both understanding and generation tasks. Compared to prior UMMs with decoupled representations, TUNA's unified visual space avoids representation format mismatches introduced by separate encoders, outperforming decoupled alternatives in both understanding and generation. Moreover, we observe that stronger pretrained representation encoders consistently yield better performance across all multimodal tasks, highlighting the importance of the representation encoder. Finally, in this unified setting, jointly training on both understanding and generation data allows the two tasks to benefit from each other rather than interfere. Our extensive experiments on multimodal understanding and generation benchmarks show that TUNA achieves state-of-the-art results in image and video understanding, image and video generation, and image editing, demonstrating the effectiveness and scalability of its unified representation design.

• The paper introduces a unified framework that cascades a VAE encoder with a robust semantic encoder to produce a consolidated visual token space.
• The methodology fuses diffusion and autoregressive paradigms, enabling state-of-the-art performance in image/video understanding, generation, and editing tasks.
• Ablation studies and benchmarks demonstrate that early fusion in unified representations yields superior synergy and efficiency compared to decoupled designs.

Native Unified Multimodal Modeling with TUNA

Introduction and Motivation

Unified multimodal models (UMMs) are engineered to perform both multimodal understanding and generation within a single architectural framework. The central design challenge for native UMMs lies in the formation of visual representations that are simultaneously suitable for tasks like image/video understanding (semantic comprehension) and image/video generation (high-fidelity synthesis). Conventional approaches use either decoupled representations—assigning separate encoders for understanding and generation—or unified representations that try to harmonize both modalities in a shared feature space. Tuna introduces a straightforward, yet robust methodology for truly unified visual representation by directly cascading a VAE encoder with a strong pretrained representation encoder, such as SigLIP 2.

Figure 1: Tuna architecture: native unified multimodal capabilities leveraging unified visual representations enabling image/video understanding and generation, and editing.

Architectural Design

Tuna’s architecture fuses the strengths of diffusion and autoregressive paradigms. The unified visual representation pipeline begins by encoding the raw image or video $\mathbf{X}$ with a causal VAE (e.g., Wan 2.2), generating a latent $\mathbf{x}_1$. For generation, the latent is noised according to a sampled timestep $t$ as $\mathbf{x}_t = t\mathbf{x}_1 + (1-t)\mathbf{x}_0$, with $\mathbf{x}_0 \sim \mathcal{N}(0,1)$. The (noised or clean) VAE latent is then subjected to a modified, robust semantic encoder (e.g. SigLIP 2) to extract high-level visual representations, followed by a lightweight MLP connector producing the unified token sequence.

Text tokens and unified visual tokens are concatenated and passed into an LLM decoder (Qwen-2.5) where causal (for text) and bidirectional (for visual) attention masks are used (see Figure 2). For image/video generation and editing, noise is introduced on visual tokens, enabling a flow-matching-based diffusion head to achieve high perceptual fidelity.

Figure 3: Overview of Tuna: VAE + representation encoder produces unified visual tokens which, joined with text, feed into the LLM decoder supporting both autoregressive understanding and flow-matching generation.

Figure 2: Attention masks in the LLM decoder for understanding and generation tasks; visual tokens are noised for generative objectives.

For video, Tuna attenuates computational overhead by applying window-based attention, processing four-frame sub-windows in parallel using the representation encoder before reaggregating as sequence tokens.

Training Pipeline

Tuna is trained in three stages:

1. Unified representation and flow-matching head pretraining: Representation encoder and flow-matching head are optimized on image captioning and text-conditioned generation. LLM decoder is frozen.
2. Full model continued pretraining: The LLM decoder is unfrozen for joint pretraining on image/video captioning, instruction-following, editing, and generation tasks.
3. Supervised fine-tuning (SFT): The model is refined on instruction-following and high-quality editing/generation datasets for final capability enhancement.

A comprehensive ablation demonstrates that joint multitask training allows bidirectional information flow, and mutual enhancement between understanding and generation, uniquely enabled by the unified representation space.

Representation Analysis and Comparison

The unified representation is critical in balancing semantic richness for understanding and fidelity for generation. Tuna’s fusion, via semantic encoder over VAE latents, performs deep feature fusion across all representation layers. In contrast, Show-o2 merges outputs from parallel branches via late fusion, leading to unbalanced features heavily biased toward semantic understanding but suboptimal for generation.

Figure 4: Illustration of representation production: Tuna (early fusion with semantic encoder over VAE latents), Show-o2 (late fusion of parallel branches).

Representation alignment via CKNNA to strong semantic/generative models confirms that Tuna achieves both high semantic similarity (for understanding) and generative feature alignment, whereas Show-o2 exhibits detrimental bias.

Figure 5: Alignment to SigLIP 2—unified representations show strong semantic feature correlation.

Performance Evaluation

Tuna achieves state-of-the-art numerical results across multimodal understanding and generation tasks:

• Image understanding: Tuna (7B) achieves 63.9 on GQA, 61.2 on MMStar, and 74.3 on OCRBench, outperforming Show-o2 and decoupled/composite UMMs.
• Image generation: Tuna consistently achieves 0.90 on GenEval, superior scores on DPG-Bench and OneIG-Bench, especially excelling in compositionality and text rendering.
• Image editing: On ImgEdit-Bench, Tuna obtains an overall score of 4.31, surpassing all tested unified models and matching dedicated generative systems.
• Video understanding/generation: On MVBench and Video-MME, Tuna (1.5B) achieves 54.4 and 49.1 respectively. On VBench video synthesis, it attains a total score of 84.06, exceeding competitive UMMs.

  Figure 6: Qualitative comparison—Tuna demonstrates precise prompt following, accurate compositionality, and strong text rendering, outperforming baselines across diverse image generation instructions.

  

  Figure 7: Qualitative comparison—robust editing performance including style transfer, object/environment change, and nuanced operations in multiple scenes.

  

  Figure 8: High-fidelity video generation with Tuna across diverse instructions, demonstrating generalization and robustness in unified representation space.

Ablations & Encoder Selection

Ablation studies confirm that unified representations built with performant semantic encoders (SigLIP 2, DINOv3) consistently yield stronger multimodal performance than decoupled or discrete designs. Early fusion with a semantic encoder over VAE latents is always preferable to late fusion or MoE-style routing, as it enables bidirectional task synergy. Further, stronger pretrained representation encoders lead to monotonic improvements across all tasks.

Implications and Prospects

Tuna’s success underscores several practical and theoretical implications:

• Efficiency: Unified representations streamline training and inference complexity compared to MoE or dual-path designs.
• Synergy: Joint representation space allows cross-task mutual enhancement, crucial for scaling UMMs that avoid catastrophic interference.
• Scalability: Early fusion enables generalization to new modalities (e.g., video), longer context, and more complex reasoning/generation tasks.
• Encoder selection: Pretraining strong semantic encoders, specifically tailored for multimodal integration, will remain a vital aspect of future unified architectures.
• Directions for future work: Extending unified tokenization to include temporal or audio modalities, improving representation learning objectives for higher compositionality, and further analyzing inductive biases from architectural choices are promising avenues.

Conclusion

Tuna establishes a new reference for native unified multimodal models by cascading a VAE encoder with a high-capacity representation encoder to produce a robust unified visual token space. Evidence from both numerical benchmarks and representation analysis demonstrates that early fusion and joint multitask training outperform decoupled, discrete, or late-fusion alternatives. Tuna’s approach yields strong state-of-the-art results in multimodal understanding, generation, and editing, marking a significant advance in scalable, efficient, and synergistic multimodal modeling.