Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models

Abstract: LLMs have achieved remarkable reliability and advanced capabilities through extended test-time reasoning. However, extending these capabilities to Multi-modal LLMs (MLLMs) remains a significant challenge due to a critical scarcity of high-quality, long-chain reasoning data and optimized training pipelines. To bridge this gap, we present a unified multi-agent visual reasoning framework that systematically evolves from our foundational image-centric model, Insight-V, into a generalized spatial-temporal architecture, Insight-V++. We first propose a scalable data generation pipeline equipped with multi-granularity assessment that autonomously synthesizes structured, complex reasoning trajectories across image and video domains without human intervention. Recognizing that directly supervising MLLMs with such intricate data yields sub-optimal results, we design a dual-agent architecture comprising a reasoning agent to execute extensive analytical chains, and a summary agent to critically evaluate and distill final outcomes. While our initial framework utilized Direct Preference Optimization (DPO), its off-policy nature fundamentally constrained reinforcement learning potential. To overcome these limitations, particularly for long-horizon video understanding, Insight-V++ introduces two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness. Crucially, by leveraging reliable feedback from the summary agent, we guide an iterative reasoning path generation process, retraining the entire multi-agent system in a continuous, self-improving loop. Extensive experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks while preserving strong capabilities on traditional perception-focused tasks.

• The paper introduces a multi-agent framework that separates visual reasoning from answer summarization, using structured trajectory generation, multimodal judging, ST-GRPO, J-GRPO, and self-evolving refinement.
• Insight-V++ achieves 71.1 average reasoning on general image benchmarks, 53.9 on advanced image reasoning, and 54.2 on video reasoning, while keeping perception performance nearly unchanged.
• The results show that jointly retrained reasoning and summary agents outperform direct supervised reasoning and inference-only collaboration, though video evaluation still depends on proprietary-model-assisted supervision and large training sets.

Motivation and problem setting

The paper addresses the difficulty of extending test-time long-chain reasoning—well established in text-only LLMs through Chain-of-Thought prompting and RL-trained systems such as OpenAI o1—to multimodal LLMs (MLLMs). Two bottlenecks are identified: the scarcity of high-quality, structured long-chain visual reasoning data (which is expensive to annotate manually), and the absence of training pipelines that let MLLMs reason in detail without degrading core perception. The work is an extension of the conference paper Insight-V [(Anjali et al., 23 Mar 2025)-style lineage; published as "Insight-V" at CVPR 2025], generalizing it from static images to video and from off-policy preference optimization to on-policy reinforcement learning.

System overview

The framework consists of three components. First, a progressive data generation pipeline samples $N$ structured reasoning trajectories per query in JSON format, where each step contains a summary, a detailed response, and an action (continue or summary). Second, a multi-granularity assessment stage filters trajectories by answer correctness using a strong LLM and then scores surviving paths (1–100) with a multimodal judge that evaluates step-wise accuracy and detail level, processing all candidates for a question in a single pass for scoring consistency. Third, a dual-agent architecture derived from a single base model: a reasoning agent trained on the highest-scoring trajectory per question to produce multi-step reasoning, and a summary agent trained on a mixture of optimal and deliberately flawed reasoning paths (sampled across score ranges) plus standard QA data, so that it critically evaluates rather than copies the reasoning chain.

For video, the authors extend this pipeline but concede a key difficulty: open-source judges cannot reliably assess video reasoning quality. Their remedy is an in-context scoring strategy: a human-verified golden set of video reasoning exemplars spanning general understanding, temporal grounding, causal reasoning, and fine-grained analysis—annotated by Gemini-2.5-Pro—is supplied as in-context examples to Qwen2.5-VL during scoring. This is an explicit dependence on proprietary frontier models both for exemplar creation and flaw annotation; the quality ceiling of the resulting dataset is therefore bounded by those teachers.

Training pipeline: from iterative DPO to GRPO variants

The original Insight-V uses two-stage SFT followed by iterative DPO, generating fresh preference pairs each round to approximate on-policy optimization. The journal version argues that DPO's off-policy nature fundamentally limits its RL potential and replaces it with two role-specific GRPO objectives:

• ST-GRPO trains the reasoning agent with a composite reward $$\mathcal{R} = 0.9\,\mathcal{R}_{task} + 0.1\,\mathcal{R}_{format}$$, where $\mathcal{R}_{task}$ combines binary answer correctness, IoU-based temporal grounding reward, and a Visual-Jigsaw-style clip reordering reward averaging per-clip position accuracy.
• J-GRPO trains the summary agent jointly on judging ($\mathcal{R}_{judge}$) and answering ($\mathcal{R}_{answer}$) over reasoning paths binned into five quality levels, with a curriculum: early stages weight the two equally on high-quality paths; later stages shift to a 3:7 ratio favoring $\mathcal{R}_{answer}$ on degraded paths. The stated rationale is that severely flawed paths are too difficult for the reasoning agent to repair, so robust final answering matters more than fine-grained judgment late in training.

Self-evolving loop

After RL, the system enters a closed loop: the reasoning agent produces a trajectory, the summary agent critiques it and answers, the reasoning agent revises conditioned on feedback (capped at three iterations), and the filtered outputs are used to re-fine-tune both agents before another RL phase. This distinguishes Insight-V++ from prior reasoner–critic systems such as Critic-V, which coordinate via prompting without joint retraining. The claim of annotation-free scaling rests entirely on the summary agent's judgments being reliable enough to serve as a filter—a circularity the paper mitigates, but does not formally bound, through the J-GRPO curriculum.

Empirical results

On general image benchmarks (MMMU, MMMU-Pro, MMBench, ChartQA, MMStar, MathVista, plus perception tasks MME, TextVQA, OCRBench, AI2D):

Configuration	Reasoning avg	Perception avg
LLaVA-NeXT baseline	46.8	65.5
+ Insight-V (iterative DPO)	54.9	72.3
Qwen2.5-VL baseline	66.3	82.9
+ Multi-Agent (RL)	69.5	82.6
+ Self-Evolving (Insight-V++)	71.1	82.8

The LLaVA-NeXT integration yields +8.1% average reasoning gain; on the stronger Qwen2.5-VL base, +4.8% additional reasoning improvement with perception essentially preserved. On advanced image reasoning (MathVision, MathVerse, WeMath, LogicVista, DynaMath, CharXiv-Reasoning), Insight-V++ reaches an average of 53.9, exceeding OpenMMReasoner (52.9) and Open-Vision-Reasoner (50.7) among 7B-scale RL-based models, with self-evolution contributing more on hard benchmarks (+2.4%) than on general ones (+1.6%).

On video reasoning (VideoMME, VideoMMMU, MMVU, VideoMMLU, VideoHolmes, Video-TT), the multi-agent RL stage lifts Qwen2.5-VL from 47.3% to 52.9%, and self-evolution adds a further 1.3% to reach 54.2% (+6.9% total). Notable per-benchmark gains include VideoMMLU 37.5→48.4 (nearly matching GPT-4o's 49.4), VideoMMMU 47.4→57.6 (surpassing Video-R1's 52.4), and Video-TT 46.8 versus GPT-4o's 46.6. These results support the central claim that reasoning–summarization decomposition transfers across modalities.

Ablations

The design-choice ablations show the multi-agent decomposition outperforming direct SFT CoT (62.1 vs 60.6 average), multi-turn supervised formats (61.0), and summary-agent-only configurations (59.8). For DPO, model-generated rationale pairs beat subsampled RLAIF-V data (0.6% vs 0.2%), and two extra DPO rounds add another 0.6%. For Insight-V++, ST-GRPO accounts for the largest share of gains (e.g., MathVision 38.7→42.9), J-GRPO adds further improvements, and self-evolving training beats a collaborative-inference-only baseline (fixed agents refining at test time), e.g., VideoMMMU 53.9 vs 55.8. Data-scaling experiments indicate the reasoning agent underperforms the baseline below ~50K training samples but improves monotonically up to 200K—an important caveat that the approach requires substantial data volume to be beneficial.

Limitations and open questions

Several constraints are acknowledged or implicit. The video data assessment depends on human-verified golden exemplars and Gemini-2.5-Pro annotations, so full autonomy from human labor is not achieved in the video domain. The self-evolving loop caps collaborative refinement at three iterations for efficiency, leaving unexamined whether deeper iteration saturates or degrades. The summary agent's reliability as the sole quality gate in self-evolution is validated only empirically; no analysis quantifies its judgment error rate or how errors propagate across evolution rounds. Finally, evaluation uses 64 frames while training supports up to 128, and all experiments are confined to ~7B-scale backbones, leaving scale-dependence of the GRPO variants open.

Conclusion

Insight-V++ consolidates a scalable reasoning-data pipeline, a reasoning/summary agent decomposition, role-specific GRPO objectives (ST-GRPO, J-GRPO), and a closed-loop self-evolving training strategy into a unified image-and-video framework. It delivers consistent state-of-the-art results among comparable open-source models—71.1 average reasoning on general image benchmarks, 53.9 on advanced reasoning, and 54.2 on video reasoning—while preserving perception, and demonstrates that tightly coupled, jointly retrained agent collaboration yields larger gains than inference-time coordination alone.