Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

Abstract: Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the two tasks and preventing fully end-to-end optimization from raw pixels. We introduce Tuna-2, a native unified multimodal model that performs visual understanding and generation directly based on pixel embeddings. Tuna-2 drastically simplifies the model architecture by employing simple patch embedding layers to encode visual input, completely discarding the modular vision encoder designs such as the VAE or the representation encoder. Experiments show that Tuna-2 achieves state-of-the-art performance in multimodal benchmarks, demonstrating that unified pixel-space modelling can fully compete with latent-space approaches for high-quality image generation. Moreover, while the encoder-based variant converges faster in early pretraining, Tuna-2's encoder-free design achieves stronger multimodal understanding at scale, particularly on tasks requiring fine-grained visual perception. These results show that pretrained vision encoders are not necessary for multimodal modelling, and end-to-end pixel-space learning offers a scalable path toward stronger visual representations for both generation and perception.

• The paper demonstrates that encoder-free, pixel embedding architectures can outperform traditional vision encoders on multimodal understanding and generation tasks.
• It employs a masking-based scheme and an x-prediction loss paradigm to train a unified transformer model end-to-end, achieving state-of-the-art performance on VQA and image synthesis benchmarks.
• The study reveals that eliminating handcrafted vision inductive biases enhances scalability, enabling refined image reconstruction and editing capabilities.

Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation

Introduction

Unified multimodal models (UMMs) have typically leveraged pretrained vision encoders—such as VAEs or semantic representation encoders—as critical frontends for processing visual data. However, this modular paradigm introduces a representational gap between image understanding and generation, complicating end-to-end optimization and imposing inductive biases that can restrict access to fine-grained visual information. In "Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation" (2604.24763), the authors propose Tuna-2, a native UMM that operates entirely in pixel space and omits all pretrained visual encoders. By employing only patch embedding layers as the interface from raw images to the model, Tuna-2 demonstrates that direct pixel-space unified modeling can match or even surpass encoder-dependent frameworks on state-of-the-art benchmarks for both multimodal understanding and generative tasks.

Figure 1: Evolution of Tuna-2 architecture and multimodal performance comparison, highlighting the removal of VAE and representation encoder in Tuna-2.

Architectural Simplification and Pixel-Space Modeling

The transition from legacy UMMs to Tuna-2 is progressive:

• Tuna-R as an Intermediate: Tuna-R eliminates the VAE and uses only a pretrained representation encoder (e.g., SigLIP 2) whose output visual tokens are fused with text in a transformer decoder.
• Tuna-2 Encoder-Free: Tuna-2 bypasses all pretrained vision encoders, directly embedding raw image patches into model tokens with simple, learnable layers. This results in a monolithic architecture with a single transformer backbone for both vision and language.

This design discards all hand-crafted visual inductive biases, permitting the model to learn image representations exclusively via large-scale multitask training. Notably, removing the encoder increases the difficulty of learning semantically meaningful and generative pixel representations due to the high-dimensionality and redundancy of pixel space.

Figure 2: Tuna-2 can perform high-fidelity text-to-image synthesis and image editing in a fully encoder-free paradigm.

Masking-Based Pixel Feature Learning

A core challenge in pixel-space modeling is enforcing robust information extraction required for both semantic understanding and image synthesis. The authors introduce a masking-based scheme: during training, image tokens are randomly masked and replaced by a learnable embedding. For generation, the model recovers masked regions conditioned on context, while for understanding, it predicts language responses from partially occluded images. This approach regularizes learning and encourages the emergence of more generalizable pixel-space features.

Figure 3: Masking-based pixel feature learning injects strong regularization during training for robust representation formation.

End-to-End Multimodal Training via Pixel-Space Flow Matching

Tuna-2 adopts an $x$-prediction/$v$-loss paradigm for image generation, inspired by recent advances in pixel-space flow matching. Given a noisy linear blend between an image and random Gaussian noise, the model is trained to predict the clean image directly. This enables generation entirely without VAE latents, aligning both understanding and generative tasks in the same pixel token space.

Experimental Evaluation

Multimodal Understanding and Visual Reasoning

Tuna-2 achieves best-in-class accuracy among 7B-scale models on a suite of nine VQA and reasoning benchmarks, including fine-grained pixel-centric tasks such as V*, CountBench, and VisuLogic. Notably, Tuna-2 outperforms both Tuna-R and previous latent-space UMMs, highlighting the benefits of removing representation encoder inductive biases and enabling direct learning from pixels at scale.

High-Fidelity Image Generation and Editing

Despite operating without latent-space bottlenecks, Tuna-2 is competitive with the strongest contemporary generative models on GenEval and DPG-Bench, achieving strong metric results across object compositionality, attribute, and alignment dimensions. While Tuna-R (encoder-based) holds a narrow lead in image generation metrics, Tuna-2 excels in diversity as judged by large multimodal LLMs.

Image Reconstruction Capability

Following lightweight tuning for image reconstruction, both Tuna-2 and Tuna-R demonstrate top-tier performance among unified tokenizers, rivaling specialized VAEs for pixel-level fidelity. Notably, Tuna-2's output is substantially better than prior non-KL-regularized autoencoder UMMs.

Figure 4: Qualitative image reconstructions for different visual tokenizers, underscoring Tuna-2’s strong reconstructive abilities in pixel-space.

Training Dynamics and Ablations

Pretraining ablations reveal that while the encoder-based Tuna-R converges more rapidly on understanding tasks, the encoder-free Tuna-2 ultimately overtakes it as training data scale increases, reflecting greater scalability and eventual superiority of end-to-end pixel-space learning. The application of masking during later pretraining stages yields consistent improvements for both understanding and generation, with a notably greater effect in Tuna-2.

Figure 5: MSE and CE pretraining loss curves under varying understanding-to-generation ratios. A 7:3 ratio yields optimal performance trade-off.

Figure 6: Model accuracy scaling as a function of training data (tokens) highlights Tuna-2's eventual outperformance on understanding tasks after sufficient pretraining.

Qualitative Attention Alignment

Visualizations of cross-modal attention show that Tuna-2 produces more precise and semantically accurate vision-language binding, even under misleading linguistic or visual distractors. Attention maps are consistently less dispersed and better focused on relevant regions compared to encoder-based architectures.

Figure 7: Attention map visualizations comparing Tuna-2, Tuna-R, and encoder-based LMMs on representative prompts and images.

Theoretical and Practical Implications

The strong performance of Tuna-2 on both understanding and generation in the absence of vision encoders challenges longstanding assumptions about the necessity of modular, pretrained visual frontends in large-scale UMMs. These results suggest that unified, monolithic transformer designs, suitably regularized via masking and trained over large-scale data, can learn the requisite low- and mid-level pixel statistics for generation as well as sufficiently abstract semantic constructs for reasoning. The empirical findings support a trend towards fully end-to-end multimodal AI and indicate that pixel-level modeling is not only tractable but also advantageous in settings demanding fine-grained visual discrimination.

Future Directions

While Tuna-2 closes the performance gap with encoder-based models in understanding and approaches their generation quality, several open questions remain. Future research could investigate:

• Scaling up encoder-free architectures to even larger model and data regimes to test their limits on ultra-high-resolution visual tasks.
• Extending the masking-based feature learning paradigm with curriculum or adaptive policies tailored to task characteristics.
• Exploring transfer to other modalities (e.g., video, audio) with minimal hand-crafted encoding fronts.

Moreover, the demonstrated flexibility and competitive efficiency of Tuna-2 suggest that pixel embedding regimes may soon become the new default foundation for unified multimodal modeling.

Conclusion

Tuna-2 establishes that encoder-free, pixel-space UMMs can match or surpass prior modular approaches across multimodal understanding, generation, and editing tasks at scale. The findings indicate that pretrained vision encoders are no longer a prerequisite for state-of-the-art multimodal AI, and that direct pixel embedding is a scalable, effective paradigm for universal visual-linguistic modeling. These advances have substantial implications for the future of model design in unified AI systems, encouraging further pivot towards end-to-end architectures without reliance on intermediate representation bottlenecks.