Video-MTR: Reinforced Multi-Turn Video Reasoning

• Video-MTR is a reinforced multi-turn reasoning framework that iteratively selects video segments and refines question comprehension for long video understanding.
• It employs an end-to-end reinforcement learning approach with a gated bi-level reward structure to optimize both answer accuracy and intermediate segment selection.
• Empirical results on benchmarks like VideoMME and MLVU demonstrate that Video-MTR achieves state-of-the-art performance using only 32 strategically selected frames.

Searching arXiv for the target paper and closely related work on long-video reasoning. Searching for "Video-MTR reinforced multi-turn reasoning long video understanding" on arXiv. Video-MTR is a reinforced multi-turn reasoning framework for long video understanding in which a multimodal LLM acts as an agent that iteratively selects video segments and refines question comprehension over multiple turns, rather than generating a prediction in a single turn. It is designed for long-form video understanding settings marked by long-range temporal dependencies, multiple events, and complex reasoning, and it is trained end-to-end without external visual-LLMs through a reinforcement-learning formulation tailored to both answer correctness and intermediate segment selection (Xie et al., 28 Aug 2025).

1. Problem formulation and conceptual scope

Long-form video understanding is presented as a setting in which events of interest may be far apart in time, critical evidence may be omitted by static sampling, and many questions require precise temporal localization together with step-wise deduction. Within this framing, Video-MTR is positioned against two broad classes of existing approaches: single-turn reasoning methods that generate answers from a uniform set of frames, and agentic pipelines that rely on external visual-LLMs and hand-engineered tools. The former risks omitting key information in long videos, while the latter is described as complex, brittle, and not end-to-end trainable (Xie et al., 28 Aug 2025).

The central claim of Video-MTR is that long-video reasoning benefits from iterative evidence acquisition. Instead of treating frame selection as a fixed preprocessing step, the framework lets the model progressively inspect additional temporal intervals based on the evolving interpretation of previously observed frames and the current question. This design is intended to combine human-like iterative reasoning with end-to-end learning, so that both segment selection and answer generation are optimized within a single trainable agent (Xie et al., 28 Aug 2025).

A useful way to situate the method is by contrast with adjacent video-language tasks. TR-DETR addresses joint video moment retrieval and highlight detection given a natural language query (Sun et al., 2024); MLVTG targets video temporal grounding and highlight detection with a Mamba-based alignment pipeline (Zhu et al., 10 Jun 2025); MTTR addresses referring video object segmentation with multimodal transformers (Botach et al., 2021). Video-MTR instead targets long-video question answering and reasoning, with the retrieval action embedded inside a multi-turn policy rather than treated as a separate temporal localization task.

2. Multi-turn interactive architecture

Video-MTR builds on the Qwen2.5-VL-7B multimodal LLM as its core MLLM (Xie et al., 28 Aug 2025). The model interacts with a video environment through a multi-turn loop. At each turn, it observes a temporal set of frames, conditions on the current question and the interaction history, and decides whether to retrieve more targeted frames by specifying a temporal interval or to answer directly.

The paper defines the current state at turn $k$ as

$$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$

where $x$ is textual input, $\mathcal{F}$ is the set of frames, and $y$ is the model output (Xie et al., 28 Aug 2025). The full sequence of states, decisions, and observations constitutes a trajectory,

$\tau = \{ (\mathcal{F}_k, x_k, y_k) \}_{k=0}^{K}.$

Operationally, the agent begins with uniformly sampled frames from the video. It then uses the accumulated evidence to request denser frames from a selected temporal interval or to terminate the interaction by emitting an answer. This progressive context update is one of the defining features of the framework: reasoning is explicitly conditioned on newly acquired evidence rather than being restricted to a fixed frame budget determined in advance (Xie et al., 28 Aug 2025).

The paper characterizes this procedure as iterative key segment selection. The intended effect is to let the model first obtain a global survey of the video and then zoom in on potentially informative intervals as hypotheses about the question are formed and revised. A plausible implication is that the architecture does not merely reallocate computation across turns; it changes the representational problem by making temporal evidence acquisition part of the policy itself.

3. Reinforcement learning objective and gated bi-level rewards

A major contribution of Video-MTR is its reward design. The paper argues that standard reinforcement learning for MLLMs typically rewards only final answer accuracy, which creates sparse credit assignment over multi-step trajectories, particularly when intermediate segment selections are critical (Xie et al., 28 Aug 2025). To address this, Video-MTR introduces a gated bi-level reward system.

The trajectory-level reward encodes answer correctness:

$$R_{acc} = \begin{cases} 1 & \text{if answer correct} \ 0 & \text{otherwise}. \end{cases}$$

The turn-level reward evaluates whether a retrieved frame set is more relevant to the question than the initial uniform selection, using IoU with the ground-truth relevant segment:

$$R_{fms}^k = \begin{cases} 0.5 & \text{if } \operatorname{IoU}(\mathcal{F}_k, \mathcal{G}) > \operatorname{IoU}(\mathcal{F}_0, \mathcal{G}) \ 0 & \text{otherwise}. \end{cases}$$

Here, $\mathcal{G}$ denotes the ground-truth relevant segment (Xie et al., 28 Aug 2025).

The critical mechanism is goal-gating. Intermediate rewards are granted only when the final trajectory answers correctly, and only one turn-level reward per trajectory is used. The reward expression is written as

$$R(\tau) = R_{acc} + \mathbf{1}_{\{R_{acc}>0\}} \cdot \max_{k=0}^{N} \bigl(R_{fms}^k + R_{\text{format}}^k\bigr).$$

The paper states that this prevents “reward hacking,” namely trajectories that repeatedly retrieve relevant frames for side reward while failing to optimize question answering (Xie et al., 28 Aug 2025).

Policy optimization is carried out with Proximal Policy Optimization (PPO). The stated objective is to maximize expected reward over trajectories,

$$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$0

with token-level advantages and both cross-turn and within-turn discounting. The summary gives the PPO ratio and clipped objective as

$$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$1

and

$$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$2

It further specifies a turn-dependent reward assignment of the form

$$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$3

This formulation couples answer accuracy with segment-selection quality instead of optimizing them independently (Xie et al., 28 Aug 2025).

4. Data curation, training procedure, and interaction protocol

Video-MTR is trained with a compact, high-quality dataset of 8K temporally grounded QA pairs (Xie et al., 28 Aug 2025). These pairs are constructed by filtering and converting from datasets such as NExT-GQA and QVHighlights. The paper emphasizes precise feedback rather than massive training scale, and contrasts this with open models trained on much larger corpora, stating that Video-MTR uses approximately 8K finely supervized instances versus 100K–4M for other open models (Xie et al., 28 Aug 2025).

The training process includes exploration bootstrapping. When the agent rarely requests more frames, a phased-out exploration bonus reward is added to encourage multi-turn evidence seeking. The stated purpose is to ensure the emergence of multi-stage behavior even under pure reinforcement learning (Xie et al., 28 Aug 2025).

The optimization environment is described concretely. Training is performed with PPO on the VAGEN framework using the Qwen2.5-VL-7B backbone and 8×NVIDIA A800-80GB GPUs. The interaction horizon is limited to 3 turns per trajectory. The first turn typically uses 16 frames, and subsequent turns use up to 8 frames each, for a total of at most 32 frames. Prompt design includes explicit instruction and interaction protocol, with natural language “thought” tokens and structured <retrieve> and <answer> actions (Xie et al., 28 Aug 2025).

These design choices establish an important property of the framework: Video-MTR is not merely a larger-frame baseline. The reported system is constrained to 32 strategically selected frames, and its gains are attributed to adaptive selection and policy learning rather than brute-force context expansion (Xie et al., 28 Aug 2025).

5. Empirical performance and ablation results

The framework is evaluated on VideoMME, MLVU, and EgoSchema, which the paper characterizes respectively as a benchmark covering short, medium, and long videos, a multi-task long-video benchmark with videos up to hours, and an egocentric video QA benchmark (Xie et al., 28 Aug 2025).

Model	Size	Frames	VideoMME-Long	VideoMME-All	MLVU-Test	EgoSchema
Video-MTR	7B	32	51.0	59.0	48.4	62.4

On these benchmarks, the paper states that Video-MTR outperforms existing methods in both accuracy and efficiency and advances the state of the art (Xie et al., 28 Aug 2025). It further reports that the method surpasses all open-source baselines in long-video regimes by large margins, including a $$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$4 improvement on VideoMME-Long and up to $$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$5 on complex multi-detail MLVU tasks. It is also described as matching or approaching proprietary models such as GPT-4o and Gemini-1.5-Pro while using only a fraction of their data and compute (Xie et al., 28 Aug 2025).

The paper also emphasizes frame efficiency: Video-MTR is reported to outperform models using 128–384 frames while using only 32 strategically selected frames (Xie et al., 28 Aug 2025). This point is central because it supports the paper’s broader claim that reasoning policy and temporal evidence selection can compensate for much larger static visual budgets.

Ablation studies are used to isolate the role of the method’s two defining ingredients. First, multi-turn reasoning outperforms single-turn reasoning by up to $$s_{k} = (\mathcal{F}_{k-w}, x_{k-w}, y_{k-w}, \ldots, \mathcal{F}_{k-1}, x_{k-1}, y_{k-1}, \mathcal{F}_k, x_k),$$6 for long videos, with benefits increasing with task complexity and duration. Second, the bi-level reward is described as essential: removing it reduces accuracy by 4–5% on the hardest benchmarks. The paper identifies goal gating as especially important, reporting that without it the model may maximize intermediate rewards at the expense of lower QA accuracy, again described as “reward hacking” (Xie et al., 28 Aug 2025).

Case studies are said to show a human-like reasoning pattern in which the model first samples global uniform frames, formulates a hypothesis, then requests frames from a suspicious interval to verify it before answering. The method is further reported to be robust across event recognition, role identification, goal reasoning, and related tasks, and to generalize to egocentric videos without being explicitly trained on ego data (Xie et al., 28 Aug 2025).

6. Interpretation, misconceptions, and relation to adjacent research

One common misconception is to treat Video-MTR as a generic temporal grounding model because it contains an explicit retrieval action. The paper’s task formulation suggests otherwise: the retrieval action is subordinate to question answering and is optimized through reinforcement learning as part of the reasoning trajectory, not as a standalone localization endpoint (Xie et al., 28 Aug 2025). This differs structurally from methods such as TR-DETR, which jointly solve moment retrieval and highlight detection through a task-reciprocal transformer (Sun et al., 2024), and from MLVTG, which tackles video temporal grounding through Mamba-based temporal modeling and LLM-driven semantic purification (Zhu et al., 10 Jun 2025).

A second misconception is that Video-MTR is simply an agentic pipeline wrapped around an external visual-LLM. The paper argues the opposite: it explicitly eliminates the need for external VLMs and engineered pipelines, with all segment search, selection, and reasoning handled within the same trainable agent (Xie et al., 28 Aug 2025). In this respect, the method participates in a broader movement toward end-to-end video-language architectures. MTTR, for example, similarly emphasizes an end-to-end multimodal transformer for referring video object segmentation rather than multi-stage post-processing pipelines (Botach et al., 2021). The difference is that Video-MTR applies this end-to-end principle to sequential decision-making over temporal evidence.

The nomenclature can also be misleading. “Video-MTR” is unrelated to MTVR, the multilingual moment retrieval dataset and its mXML model for English-Chinese video moment retrieval (Lei et al., 2021), and it is likewise unrelated to motion-prediction works using the acronym MTR in autonomous driving (Shi et al., 2023). In the long-video understanding literature, Video-MTR specifically denotes reinforced multi-turn reasoning for question answering over long videos (Xie et al., 28 Aug 2025).

Taken together, the reported results suggest a shift in emphasis from static frame allocation toward learned sequential evidence acquisition. A plausible implication is that future long-video systems may increasingly treat temporal observation as an adaptive decision problem rather than as a fixed sampling problem. In the formulation presented by Video-MTR, long-video understanding becomes a joint problem of policy learning, temporal evidence selection, and multimodal reasoning, trained end-to-end under a reward structure that couples intermediate retrieval behavior to final task success (Xie et al., 28 Aug 2025).