Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification (2506.07235v1)

Abstract: Multi-modal LLMs (MLLMs) have achieved remarkable capabilities by integrating visual perception with language understanding, enabling applications such as image-grounded dialogue, visual question answering, and scientific analysis. However, most MLLMs adopt a static inference paradigm, encoding the entire image into fixed visual tokens upfront, which limits their ability to iteratively refine understanding or adapt to context during inference. This contrasts sharply with human perception, which is dynamic, selective, and feedback-driven. In this work, we introduce a novel framework for inference-time visual token scaling that enables MLLMs to perform iterative, verifier-guided reasoning over visual content. We formulate the problem as a Markov Decision Process, involving a reasoner that proposes visual actions and a verifier, which is trained via multi-step Direct Preference Optimization (DPO), that evaluates these actions and determines when reasoning should terminate. To support this, we present a new dataset, VTS, comprising supervised reasoning trajectories (VTS-SFT) and preference-labeled reasoning comparisons (VTS-DPO). Our method significantly outperforms existing approaches across diverse visual reasoning benchmarks, offering not only improved accuracy but also more interpretable and grounded reasoning processes. These results demonstrate the promise of dynamic inference mechanisms for enabling fine-grained, context-aware visual reasoning in next-generation MLLMs.

• The paper introduces a dynamic visual tokens scaling framework that integrates an iterative verifier-guided reasoning process to enhance visual inference in MLLMs.
• It formulates the problem as a Markov Decision Process with distinct reasoner and verifier components, validated on benchmarks such as BLINK and V*Bench.
• Experimental results demonstrate significant gains in complex visual tasks, offering more interpretable and reliable reasoning compared to traditional models.

Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification

The paper "Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification" explores the challenges and advancements in the domain of multi-modal LLMs (MLLMs) with a focus on enhancing their visual reasoning capabilities. Unlike traditional MLLMs that embody a static and monolithic approach to visual understanding, this research proposes a dynamic framework that aims to emulate human-like perception which is iterative and feedback-driven.

Core Contributions and Methodology

The key contribution of this paper is the introduction of an inference-time visual token scaling framework that integrates an iterative verifier-guided reasoning process. This proposed method allows MLLMs to perform multi-step visual reasoning, an aspect that has been relatively underexplored in previous iterations of MLLMs.

1. Markov Decision Process Framework: The authors formulate the challenge as a Markov Decision Process (MDP), consisting of two essential components:
   • Reasoner: This component is a MLLM enhanced with modular visual tools to propose visual actions.
   • Verifier: Trained using multi-step Direct Preference Optimization (DPO), the verifier assesses the quality of actions and determines the termination of the reasoning process.
2. Visual Tokens Scaling (VTS) Dataset: The VTS dataset is introduced, comprising VTS-SFT (supervised reasoning trajectories) and VTS-DPO (preference-labeled reasoning pairs). These datasets are designed to facilitate the training and evaluation of both reasoner and verifier components.
3. Experimental Validation: The superiority of the proposed method is validated across a variety of visual reasoning benchmarks such as BLINK, V*Bench, and others, where it demonstrates enhanced accuracy and offers more interpretable visual reasoning processes over existing models.

Numerical Results and Implications

Experimental evaluations indicate significant improvements in visual reasoning tasks when utilizing the VTS-V framework. Notably, in a comparison within the BLINK benchmark, the introduced method consistently outperforms baseline models like MMFactory and CoT in both closed-source (using GPT-4o) and open-source (using Qwen2-VL models) settings.

A notable observation is the substantial performance gain in complex visual tasks that necessitate an ability to discern fine details and contextual relevance, attributes that traditional static token models struggle with. This underscores the practical utility of VTS-V in applications demanding nuanced visual-understanding.

Theoretical and Practical Implications

From a theoretical standpoint, the paper extends the understanding of large-scale MLLMs, demonstrating the potential of dynamic multi-step processing for visual inference tasks. Practically, the introduction of a verifier as an integral part of the reasoning loop may lead to more reliable AI systems capable of justifying their outputs through explicit visual evidences.

Future Directions

This work opens several avenues for future research. Potential developments involve expanding the diversity and capability of visual tools integrated within the reasoner, refining the interaction mechanisms between the reasoner and verifier, and adapting the model to handle more complex multi-modal inputs. Long-term, this could contribute to the evolution of more robust and generalizable AI systems, capable of human-like exploratory and interactive perception.

In conclusion, the paper presents a seminal shift in the design of MLLMs, shifting from a static understanding framework towards a dynamic, multi-step reasoning approach, aligning closer to human cognitive processes. This not only enhances interpretability and performance in existing visual reasoning benchmarks but also sets a foundational blueprint for future research in the AI community.