DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models

Abstract: While recent Multimodal LLMs (MLLMs) have attained significant strides in multimodal reasoning, their reasoning processes remain predominantly text-centric, leading to suboptimal performance in complex long-horizon, vision-centric tasks. In this paper, we establish a novel Generative Multimodal Reasoning paradigm and introduce DiffThinker, a diffusion-based reasoning framework. Conceptually, DiffThinker reformulates multimodal reasoning as a native generative image-to-image task, achieving superior logical consistency and spatial precision in vision-centric tasks. We perform a systematic comparison between DiffThinker and MLLMs, providing the first in-depth investigation into the intrinsic characteristics of this paradigm, revealing four core properties: efficiency, controllability, native parallelism, and collaboration. Extensive experiments across four domains (sequential planning, combinatorial optimization, constraint satisfaction, and spatial configuration) demonstrate that DiffThinker significantly outperforms leading closed source models including GPT-5 (+314.2\%) and Gemini-3-Flash (+111.6\%), as well as the fine-tuned Qwen3-VL-32B baseline (+39.0\%), highlighting generative multimodal reasoning as a promising approach for vision-centric reasoning.

• The paper proposes a novel paradigm that recasts multimodal reasoning as a visual generation problem using diffusion models, enhancing logical consistency and spatial precision.
• It introduces a Multimodal Diffusion Transformer trained via ODE-based flow matching, ensuring constant inference time and robust performance on complex, vision-centric tasks.
• Experimental results show significant accuracy gains over state-of-the-art MLLMs, including improvements over GPT-5, Gemini-3-Flash, and Qwen3-VL, underscoring its practical efficiency and scalability.

DiffThinker: Advancing Generative Multimodal Reasoning with Diffusion Models

Introduction

The paper "DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models" (2512.24165) introduces a paradigm shift in multimodal reasoning by recasting the reasoning process as a native visual generation problem, specifically utilizing diffusion models. Traditional Multimodal LLMs (MLLMs) predominantly operate in the symbolic (text) space, employing chain-of-thought (CoT) reasoning and multi-turn image interaction to address vision-centric tasks. However, these methods exhibit significant limitations in complex, long-horizon, and vision-heavy problems due to inefficiencies, lack of controllability, and suboptimal handling of state transitions. DiffThinker proposes and comprehensively validates a generative multimodal reasoning framework intended to address these deficiencies, leveraging the strengths of diffusion models for image-to-image transformation.

Paradigm Shift: Multimodal Reasoning as Visual Generation

Motivation and Conceptual Reformulation

Traditional MLLMs formulate multimodal reasoning as a mapping from multimodal (typically image-plus-text) inputs to symbolic solutions, primarily through sequential, token-level outputs. Despite recent augmentation via visual feedback loops ("Thinking with Image" and "Thinking with Video" paradigms), these approaches are hampered by their inherent sequentiality, instability in inference runtime, limited parallelism, and brittleness under increased task complexity.

DiffThinker departs fundamentally from this operational regime. It reformulates multimodal reasoning as a direct image-to-image generative process, exploiting the spatial and structural coherence achievable by diffusion models. The solution itself is generated in visual space—such as trajectory paths on grids, completed Sudoku boards, or reconstructed jigsaw images—which can be objectively parsed and compared in the symbolic space for evaluation. This enables logical consistency and spatial precision that are unavailable to purely text-centric symbolic reasoners.

Model Architecture and Training Protocols

Core Framework

DiffThinker builds on state-of-the-art components: it utilizes Qwen-Image-Edit as the base and employs the Multimodal Diffusion Transformer (MMDiT) [Esser et al., 2024] for cross-modal dependencies, operating in the latent space of a VAE for efficiency. The model is trained via Flow Matching, an ODE-based method that maps noise to the data manifold, ensuring stable convergence and robust generation.

Conditioning and Objective

Inputs (image, instruction) are encoded and combined into a latent conditioning vector. For each training instance, a random timestep is sampled, a latent is linearly interpolated between noise and data, and the velocity field is regressed using MSE against the ideal path. During inference, reasoning is performed by numerically integrating the ODE from noise to data manifold, resulting in the generated solution image.

Efficiency and Hyperparameterization

The paradigm guarantees constant inference time and controllable computational budgets regardless of reasoning depth, achieved by fixing the number of ODE steps per instance. Empirical ablations identify a balance point at 20 inference steps, optimizing both accuracy and runtime. Furthermore, classifier-free guidance is employed; performance peaks at a scale of four, balancing logical fidelity and generative accuracy.

Experimental Results and Comparative Evaluation

Task Suite and Baselines

The evaluation spans seven tasks across four reasoning domains:

• Sequential Planning: Visual Spatial Planning (VSP), VSP-Super, Maze
• Combinatorial Optimization: Traveling Salesperson Problem (TSP)
• Constraint Satisfaction: Sudoku
• Spatial Configuration: Jigsaw, VisPuzzle

Baselines include closed-source SOTA models (GPT-5, Gemini-3-Flash) and open MLLMs (Qwen3-VL-8B/32B) both in SFT and RL (GRPO) regimes, all fine-tuned identically for fairness.

Numerical Results

DiffThinker exhibits superior performance, quantified by:

• +314.2% over GPT-5
• +111.6% over Gemini-3-Flash
• +39.0% over Qwen3-VL-32B

across aggregate accuracy metrics in all domains. In vision-centric long-horizon tasks (e.g. large-scale Maze, VSP-Super, high-complexity Jigsaw), DiffThinker maintains high accuracy while MLLM performance rapidly degrades with increased difficulty. Near-perfect reconstruction is achieved in spatial configuration tasks, highlighting the efficacy of image-based reasoning for directly visual problems.

Efficiency and Controllability

DiffThinker matches or exceeds leading MLLMs in both training and inference speed, despite its generative nature. Crucially, its fixed-step generative trajectory ensures deterministic inference costs, unlike variable-length CoT in MLLMs, obviating premature truncation and token collapse risks.

Native Parallelism and Collaborative Reasoning

An intrinsic property is the model’s parallel exploration of multiple reasoning paths early in the forward ODE, with progressive pruning to optimal solutions. This native parallelism contrasts starkly with the strictly sequential MLLM process and enables effective global constraint satisfaction.

The system also facilitates synergistic collaboration: generating multiple candidate solutions which are filtered by MLLMs’ symbolic reasoning, consistently yielding higher accuracy than either system alone. This highlights DiffThinker’s value as a backend visual reasoner augmenting more generalist language-based systems.

Ablations: Robustness to Steps, Data, and Guidance

• Inference Steps: Rapid saturation in accuracy suggests that the reasoning manifold is captured efficiently by the model.
• Training Data Scale: The model continues to benefit from larger data, effectively internalizing causal and structural relationships at scale—contrast to MLLMs’ plateauing.
• Classifier-Free Guidance: Optimal logical fidelity and visual quality at guidance=4, with unstable behaviour at extremes.

Image vs. Video Generation

DiffThinker-Video, implemented using open-source video diffusion models, displays inherent multimodal reasoning capacities but is outperformed by image-based DiffThinker in both accuracy and compute efficiency. Video generation incurs substantially greater computational overheads, indicating that improvements in video diffusion models are necessary before they become practical for general multimodal reasoning.

Theoretical and Practical Implications

DiffThinker formalizes and evidences the generative multimodal reasoning paradigm as a compelling alternative to symbolic, sequential reasoning in complex vision tasks. The paper’s empirical results and in-depth analysis reveal new trade-offs and capabilities:

• Fundamentally, shifting the reasoning process into the visual generative domain yields superior logical consistency and spatial precision.
• Parallelism and determinism in generation enhance scalability to task complexity, suggesting suitability for deployment in real-time or resource-constrained scenarios.
• The paradigm’s adaptability implies that diverse vision-centric tasks can be tackled with a unified architecture, streamlining model design and deployment.
• Synergistic collaboration between generative visual models and language-based models offers a promising path for extending multimodal reasoning capabilities beyond strictly vision- or language-centric domains.

Limitations and Future Directions

DiffThinker’s generalization to out-of-distribution, zero-shot scenarios is limited by the base generative model’s representational power. Future research should prioritize rethinking foundation model training with explicit focus on visual reasoning. Moreover, while DiffThinker surpasses traditional MLLMs in vision-centric contexts, the latter retain an advantage in strictly symbolic (e.g., text-based mathematics) tasks, motivating future exploration of hybrid, deeply integrated architectures that leverage both paradigms’ strengths.

Conclusion

DiffThinker demonstrates that recasting multimodal reasoning as a visual, diffusion-driven generative problem is both empirically effective and theoretically compelling for a broad range of complex, vision-centric tasks. The paradigm’s core advantages—efficiency, controllability, parallelism, and collaborative potential—establish a new direction in multimodal intelligence, with significant implications for the design and deployment of next-generation AI systems. Further advances in foundational generative modeling and deeper integration with symbolic systems are expected to unlock broader, even cross-modal, reasoning capabilities.