DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

Abstract: Current Unified Large Multimodal Models (ULMMs) support interleaved multimodal reasoning through textual reasoning and intermediate visual states, but typically generate each visual state as a full image. This full-image generation paradigm introduces substantial visual-token redundancy and dilutes supervision on sparse yet reasoning-critical state transitions. We propose DeltaV, a ULMM that replaces full-image generation with visual updates. Conditioned on historical visual states, DeltaV incrementally predicts compact update tokens that capture the visual changes across reasoning steps, avoiding repeated modeling of unchanged content. To align the token budget of each update with the magnitude of visual change, DeltaV introduces a temporal similarity (TSIM) Router, which stops allocating tokens once the marginal reconstruction gain falls below a threshold. To support more diverse and generalizable reasoning, we further construct StructCoT, a large-scale interleaved multimodal reasoning dataset with 1.05M samples spanning 44 task domains. Experiments show that the visual-update paradigm reduces newly generated visual tokens by 55.6\% on average without compromising reconstruction fidelity, and improves multimodal reasoning by 3.3\% over full-image generation. Trained with StructCoT and large-scale multimodal data, DeltaV-2B further outperforms substantially larger open-source models by 8.4\% on in-domain multimodal reasoning evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9\% on external multimodal reasoning and understanding benchmarks. Code, models, and StructCoT will be released at https://github.com/Pengjie-W/DeltaV.

• The paper proposes state-update modeling that focuses on semantic visual differences, reducing redundant tokens and enhancing multimodal reasoning efficiency.
• It introduces TSIM-Tok and TSIM-Router, a token allocation mechanism using visual similarity metrics to dynamically allocate tokens based on incremental changes.
• Experimental results demonstrate up to 55.6% token reduction, improved SSIM and PSNR, and notable gains in reasoning accuracy compared to full-image approaches.

DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

Motivation and Background

Unified Large Multimodal Models (ULMMs) have achieved significant progress in integrating visual and textual reasoning. Classic interleaved multimodal reasoning protocols depend on representing each intermediate visual state as a complete image. However, this paradigm leads to visual-token redundancy and, crucially, dilutes the learning signal for semantically meaningful, reasoning-driven visual transitions. DeltaV proposes an alternative: rather than reconstructing every visual state from scratch, it focuses on state evolution, incrementally modeling only the informative visual difference between reasoning steps. This transition-centric approach aligns with the inherently sparse change patterns in many multi-step reasoning tasks.

Figure 1: Conceptual overview contrasting existing ULMMs, which regenerate full visual states, and DeltaV, which predicts variation-aware updates dynamically allocated to critical visual differences.

The DeltaV framework is underpinned by two innovations: (1) a formalization of interleaved multimodal reasoning as reasoning over visual state updates instead of complete state reconstructions, and (2) a token allocation mechanism—via the Temporal Similarity (TSIM) Router—that tightly couples visual update size to the magnitude of visual change.

DeltaV Architecture and TSIM-Tok

DeltaV’s autoregressive generation decomposes the multimodal trajectory into initial base visual tokens and a sequence of text and variable-length visual updates:

$$Y = \{Z_0, X_1, \Delta Z_1, X_2, \Delta Z_2, ... \}$$

where $Z_0$ is the initial visual state and $\Delta Z_t$ denotes compact, adaptive update tokens for the visual change at step $t$.

TSIM-Tok, the visual tokenizer, is designed to efficiently encode these updates. Given two visual states, it computes patch-level feature similarity—weighted by both temporal proximity and previously allocated token budgets—to yield a temporal similarity (TSIM) score. This score, via offline calibrated curves, dictates the minimal required token budget per update step to maintain reconstruction fidelity while focusing on semantically relevant changes.

Figure 3: Architecture overview. DeltaV models reasoning as an initial base state followed by adaptive visual updates, with TSIM-Tok and TSIM Router orchestrating variation-aware token allocation.

To establish the mapping from TSIM value to token budget, an offline calibration process fits the relationship between token count and incremental reconstruction gains as measured by SSIM. Token allocation stops when incremental gain falls below a defined threshold, ensuring efficient use of token budget.

Figure 2: TSIM-driven token allocation pipeline. Calibration empirically links TSIM, update token count, and image fidelity, allowing dynamic assignment of token budgets per step.

StructCoT Dataset

DeltaV’s training leverages StructCoT, a large-scale dataset specifically constructed for interleaved multimodal reasoning with diverse update patterns and reasoning structures. StructCoT comprises 1.05M samples across 44 tasks, providing the coverage necessary for generalizable, update-centric multimodal reasoning.

Figure 4: Comparative task taxonomy and scale of StructCoT versus prior datasets, highlighting its broad coverage and granularity.

Figure 5: Examples of interleaved reasoning processes across seven StructCoT categories, showing ground-truth sequences of text and visual updates.

Experimental Analysis

Efficiency and Fidelity of Visual Updates

Visual-update modeling significantly improves over classic full-image modeling for both reconstruction and reasoning. Fixed-token and TSIM-Router-driven update modeling consistently achieve higher SSIM and PSNR at comparable or lower token budgets. Critically, using only 64 visual-update tokens per step matches the reconstruction quality of full-image modeling with 144 tokens but with a 55.6% reduction in newly generated visual tokens, reflecting highly efficient information use.

Figure 6: Quantitative trade-off between full-image and visual-update modeling—DeltaV achieves superior image fidelity at substantially lower token budget.

Multimodal Reasoning and Generalization

DeltaV’s visual-update approach yields a 3.3% performance gain on interleaved multimodal reasoning ablations compared to full-image models. When compared on in-domain and external benchmarks, DeltaV-2B outperforms much larger open-source models (e.g., Qwen3-VL-8B, Qwen2.5-VL-72B) by ~8.4% and achieves a 5.9% advantage over Qwen3-VL-2B on external multimodal reasoning/understanding tasks, validating the paradigm’s strong scaling properties and transfer robustness.

Ablation studies show TSIM-driven dynamic allocation enables higher token efficiency and better fidelity than random or value-based allocation baselines. The slope-based TSIM allocation is optimal, better reflecting diminishing returns as additional tokens contribute less to reconstruction quality.

Qualitative Insights

Qualitative analysis reveals that DeltaV reduces semantic drift and inconsistencies in reasoning chains, in contrast with full-image approaches that often lose previously established visual evidence, causing errors in downstream reasoning steps.

Figure 9: Qualitative reconstruction—full-image modeling suffers from semantic drift whereas update modeling maintains identity and color consistency.

Figure 7: Multimodal reasoning comparison—DeltaV preserves visual evidence across steps, avoiding cumulative errors stemming from inconsistent intermediate generations.

Theoretical and Practical Implications

DeltaV formalizes multimodal reasoning as a process of state refinement via sparse, adaptive visual updates, thus concentrating supervision on the causal transitions essential for correct reasoning. This approach directly addresses the supervision dilution and computational cost of full-image interleaved modeling.

Practically, the visual update paradigm supports more efficient, longer-horizon, and compositional reasoning by reducing redundant autoregressive overhead and enabling models to maintain tighter consistency across steps. The token allocation protocol—grounded in observable visual similarity—provides a general mechanism for balancing representation fidelity against sequence length, which can be extended to other domains where state continuity is present.

Theoretically, the paradigm shift from state reconstruction to state-transition modeling suggests broader implications for multi-modality: wherever history and causality exhibit sparse updates, update-centric supervision should outperform generative baselines focused on reconstructing complete states. Future developments may involve further automation of token allocation, integration with external tool-use for deterministic operations, and fusion of update-centric modeling with latent world models.

Conclusion

DeltaV reframes interleaved multimodal reasoning as reasoning over adaptive, sparse visual state updates, operationalized through TSIM-driven token allocation. This principled shift enables more efficient, consistent, and semantically dense multimodal reasoning, as evidenced by significant numerical advantages over traditional and state-of-the-art baselines. Alongside StructCoT, DeltaV sets a new direction for unified multimodal intelligence by making causal state transitions the central object of model supervision and reasoning.