Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models

Abstract: Unified multimodal models (UMMs) were designed to combine the reasoning ability of LLMs with the generation capability of vision models. In practice, however, this synergy remains elusive: UMMs fail to transfer LLM-like reasoning to image synthesis and exhibit divergent response behaviors. We term this phenomenon pseudo-unification. Diagnosing its internal causes is important, but existing probing methods either lack model-internal insight or ignore prompt-response dependencies. To address these limitations, we propose an information-theoretic probing framework that jointly analyzes how UMMs encode inputs and generate outputs. Applied to ten representative UMMs, our framework reveals that pseudo-unification stems from a dual divergence: (i) Modality-Asymmetric Encoding, where vision and language follow different entropy trajectories, and (ii) Pattern-Split Response, where text generation exhibits high-entropy creativity while image synthesis enforces low-entropy fidelity. Only models that unify both sides (e.g., via contextual prediction) achieve more genuine unification, enabling stronger reasoning-based text-to-image generation even with fewer parameters. Our work provides the first model-internal probing of unification, demonstrating that real multimodal synergy requires consistency in information flow, not just shared parameters.

• The paper demonstrates that pseudo-unification arises from unsynchronized reasoning across modalities in unified multimodal models.
• It employs a kernel-based entropy probing framework to quantify prompt and response encoding uncertainties across text and image modalities.
• Findings suggest that only architectures enforcing unified prediction biases achieve genuine multimodal synergy.

Entropy-Probing Reveals Pseudo-Unification in Unified Multimodal Models

Introduction

Unified Multimodal Models (UMMs) promise the convergence of the creative reasoning faculty characteristic of LLMs and the fidelity-focused capacity of vision architectures for image synthesis through a shared architecture. However, the empirical evidence presented in "Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models" (2604.10949) establishes that this unification is often superficial: current UMMs do not synchronize reasoning logics across modalities, resulting in what the authors term pseudo-unification. Addressing this, the paper introduces a kernel-based information-theoretic probing framework that is both model-internal and modality-agnostic. By systematically examining prompt and response representation entropy, the work uncovers modality-asymmetric encoding and pattern-split generative regimes that persist even in the absence of explicit architectural segregation.

A key empirical finding is that only models which architecturally unify both encoding and generative processes—specifically by enforcing contextually homogeneous prediction biases—approach genuine multimodal synergy. These insights prioritize internal consistency of information flow over simply unifying parameters.

Architectural Taxonomy of UMMs

The surveyed UMMs are categorized into two principal families: native UMMs, which implement architectural unification through a single all-in-one Transformer, and pipeline models combining MLLMs with external diffusion-based image generators. For instance, Harmon and Janus-Pro exemplify the former with joint token generation; BAGEL leverages a Mixture-of-Transformers composition, while OmniGen2 delegates visual synthesis to an auxiliary module. These structural choices directly influence information dynamics and the extent of cross-modal integration.

Figure 1: Current UMM architectures span native unified frameworks and pipeline approaches, which fundamentally shape their information alignment.

Kernel-Based Entropy Probing Framework

Quantifying information flow in UMMs is complicated by implicit, variable-length high-dimensional representations that preclude classical density-based entropy estimation. The paper overcomes this with a non-parametric, kernel-based (matrix Rényi) entropy formulation. This approach computes prompt and response embedding entropy via pairwise representation similarities, and operationalizes a proxy for conditional entropy by embedding sequence block-kernels. These metrics accurately recover known theoretical behaviors (entropy increases with representational diversity, conditional entropy rises with modality decoupling) and are robust to architecture and input heterogeneity.

Figure 2: Schematic of the entropy probing framework, enabling modality-agnostic quantification of encoding and generative uncertainties.

Empirical Findings: Prompt Representation

Experimental probing on ten state-of-the-art UMMs using varied datasets (T2I-CoReBench for text, MMBench for image) demonstrates that:

• Embedding and layer-wise entropy are predominantly dictated by architectural priors and model scale, not semantic prompt structure.
• Larger models often exhibit early-layer entropy compression for text, favoring cross-modal alignment over textural detail, while smaller models display prolonged preservation of semantic diversity.
• All models show nearly identical encoding trajectories regardless of prompt type, confirming a structure-agnostic approach.
• Systematic cross-modal asymmetry emerges: for example, in BAGEL, visual representations maintain high entropy while text is compressed, whereas Harmon reveals asynchronous convergence with language encoded more aggressively than vision.

  Figure 3: Prompt entropy trajectories reveal model- and scale-dependent behavior, largely insensitive to prompt length or type.

  

  Figure 4: Layer-wise entropy remains invariant across text prompt types, indicating structure-agnostic encoding.

  

  Figure 5: Image prompt encoding is indifferent to prompt type, again confirming architecture-driven representational geometry.

Overall, the representational spaces of text and images remain segregated in their entropy dynamics even under shared parameters, presaging divergent generative behaviors.

Empirical Findings: Response Pattern and Pseudo-Unification

Layer-wise analysis of conditional entropy between prompt and response embeddings for both text and image generations identifies the "pattern-split" characteristic of pseudo-unification. With the notable exception of Harmon, UMMs generate text with higher conditional entropy (indicative of creative, diverse outputs) and images with low conditional entropy (fidelity-dominated responses). This demonstrates that, in current architectures, creative reasoning fails to transfer from text to image generation—the origin of the pseudo-unification nomenclature.

Figure 6: Conditional entropy measurements expose the divergence in response behaviors: only Harmon achieves cross-modal convergence in conditional entropy.

Harmon's architecture, which grounds both modalities in contextual prediction (masked autoencoding for vision, next-token for language), alone achieves conditional entropy convergence between text and image in the deepest layers, thus empirically substantiating a blueprint for real unification.

Theoretical and Practical Implications

The research elucidates that explicit parameter sharing is insufficient to deliver true multimodal reasoning and that the inductive biases inherent in generation objectives (autoregressive vs. diffusion, etc.) enforce lasting modality-dependent response distributions. From a diagnostic standpoint, the introduced kernel entropy measures offer a generalizable and scalable tool for probing representational and generative unification in any implicit-modeling setting.

Practically, these findings have several consequences:

• Architectural unification must be accompanied by design choices (e.g., contextual prediction unification) that enforce shared information flow and generative logic across modalities.
• Reliance on prompt engineering to achieve information consistency is largely ineffective once a model’s representational geometry is established.
• Improvements in UMMs will require innovation in shared training objectives and structural mechanisms, beyond scaling and data curation.

Theoretically, the framework links empirical representational geometries with probabilistic generation regimes, offering a rigorous foundation for hypothesis-driven model development.

Future Directions

Moving beyond post-hoc metrics and black-box benchmarks, future work should enforce symmetry in pretraining objectives and adopt interventionist evaluations focusing on information consistency. Mechanistically, enforcing contextual prediction across modalities may be necessary and sufficient for genuine multimodal synergy.

Conclusion

The paper provides compelling evidence that current UMMs exhibit pseudo-unification: modal asymmetries pervade both encoding and generative behaviors, traceable to architecture-driven entropy misalignments and differing inductive biases. Realizing genuine unification demands architectural and objective-level calibration of information flow, a direction analytically and empirically supported through the proposed entropy probing framework (2604.10949).