MOSS-ChatV: RL Framework for Video Reasoning

• MOSS-ChatV is a reinforcement learning framework for video temporal reasoning in MLLMs, addressing process inconsistency by aligning reasoning with video dynamics.
• It uses a DTW-based process reward that matches generated reasoning traces against annotated references, achieving 87.2% accuracy on the MOSS-Video test set.
• Leveraging Group Relative Policy Optimization, the framework enhances temporal fidelity and robustness across multiple video benchmarks including MVBench and MMVU.

MOSS-ChatV is a reinforcement learning framework for video temporal reasoning in multimodal LLMs (MLLMs). It is designed to address process inconsistency, a failure mode in which intermediate reasoning drifts from video dynamics even when the final answer is correct, thereby undermining interpretability and robustness. The framework introduces a Dynamic Time Warping (DTW)-based process reward that aligns generated reasoning traces with temporally grounded references, and it is trained and evaluated with MOSS-Video, a benchmark containing annotated reasoning traces. Reported results include 87.2\% on MOSS-Video (test), together with improvements on general video benchmarks such as MVBench and MMVU (Tao et al., 25 Sep 2025).

1. Problem formulation and conceptual basis

Video reasoning in MLLMs requires models to move beyond static perception toward coherent understanding of temporal dynamics in complex scenes. Within this setting, MOSS-ChatV treats dynamic state prediction as a key measure of video reasoning. The core observation is that answer correctness alone is insufficient: a model may produce the correct endpoint while its step-by-step rationale remains temporally unfaithful to the underlying video. MOSS-ChatV is therefore centered on process-level supervision, not only outcome-level optimization (Tao et al., 25 Sep 2025).

The framework addresses a specific gap in prior reinforcement-learning approaches for multimodal reasoning. Existing systems commonly optimize final accuracy or depend on auxiliary reward models. MOSS-ChatV instead uses a rule-based reward that aligns reasoning traces with annotated references. This design is intended to provide efficient process supervision without auxiliary reward models, while preserving temporal grounding in the reasoning trajectory itself (Tao et al., 25 Sep 2025).

A common misconception is to equate correct final answers with correct video reasoning. MOSS-ChatV is explicitly motivated by the opposite observation: intermediate reasoning may be inconsistent with the depicted temporal process even when the answer is correct. In that sense, the framework reorients evaluation from terminal correctness toward the internal temporal structure of inference (Tao et al., 25 Sep 2025).

2. Reinforcement-learning framework

MOSS-ChatV uses Group Relative Policy Optimization (GRPO), described as a scalable, sample-efficient RL algorithm for LLMs and associated in the paper with DeepSeek-R1 (DeepSeek-AI et al., 22 Jan 2025). For each input, the policy $\pi_\theta$ samples $G$ responses, computes a scalar reward for each response, and standardizes the resulting advantages as

$$A_i = \frac{\mathcal{R}_i - \mu}{\sigma},\quad \mu = \operatorname{mean}(\{\mathcal{R}_i\}), \quad \sigma = \operatorname{std}(\{\mathcal{R}_i\})$$

The reward used in training is decomposed into three components: an accuracy reward, a format reward, and a process reasoning reward. The total reward is

$$\mathcal{R}_i = \mathcal{R}_{\mathrm{acc},i} + \mathcal{R}_{\mathrm{fmt},i} + \mathcal{R}_{\mathrm{proc},i}$$

The accuracy reward is $1$ if the final answer is correct and $0$ otherwise. The format reward is $1$ if the output matches the required schema, such as exposing > ... reasoning, and $0$ otherwise. The process reasoning reward is computed by aligning generated reasoning against reference reasoning (Tao et al., 25 Sep 2025).

This decomposition is important because it separates three distinct desiderata: terminal correctness, structural compliance, and temporally coherent intermediate reasoning. A plausible implication is that the framework is intended to discourage superficially correct but procedurally ungrounded outputs, while also stabilizing the format required for process-level evaluation.

3. DTW-based process reasoning reward

The distinctive component of MOSS-ChatV is its DTW-based process reward, implemented through a “split-align” procedure. Both generated reasoning and reference reasoning—the content inside <think>—are segmented into discrete reasoning steps, for example at sentence level using nltk. This yields two sequences:

$Seq_\text{gen} = \{g_1, ..., g_m\}$

$Seq_\text{ref} = \{r_1, ..., r_n\}$

A textual similarity matrix $G$0 is then constructed with entries

$G$1

where

$G$2

Rather than applying ordinary DTW in a way that penalizes any extra reasoning, the framework uses Subsequence Dynamic Time Warping (SDTW). SDTW aligns the reference reasoning as a contiguous process embedded within the generated trace, thereby ignoring extra steps outside the alignment. The method also allows controlled “jumps,” described as tolerance for minor deviations or missing steps, to balance flexibility and strictness. If the minimal alignment cost is $G$3, then the process reward is

$G$4

This construction serves several purposes at once. It rewards temporal fidelity by favoring reasoning that matches annotated steps in order; it tolerates additional valid exploration; and it is explicitly intended to avoid the reward hacking observed with naive DTW, where models can “short circuit” by generating minimal reasoning to minimize the DTW path. The paper characterizes the annotated traces as a “minimal gold standard”: extra correct reasoning is tolerated, but reward is not inflated by repetition (Tao et al., 25 Sep 2025).

4. MOSS-Video benchmark

MOSS-ChatV is coupled to MOSS-Video, a benchmark constructed from ShareGPT4Video and designed for process-level supervision in video reasoning. The dataset contains 11,654 training samples (w/ 1,218 unique videos) and 2,836 test samples (w/ 479 unique videos). Each sample includes a video, a question about future or current object states, a ground-truth answer, and step-by-step reasoning annotations (Tao et al., 25 Sep 2025).

The annotations are built around object-state dynamics. The benchmark includes both coarse-grained annotations of the form <object, state, timestamp> and fine-grained annotations of the form <object, change description, timestamp>, extracted using multiple passes of GPT-4o prompt-based annotation. Two properties are emphasized in the description of the dataset: temporal state prediction, because queries always require dynamic state understanding, and reasoning trace supervision, because each answer is paired with annotated multi-step temporal explanations (Tao et al., 25 Sep 2025).

The benchmark is also part of the training protocol. The training split is used to fine-tune MOSS-ChatV, while the held-out split is reserved for evaluation. This makes MOSS-Video both a supervised source of temporally grounded reasoning traces and an evaluation framework for held-out generalization under process-level reinforcement learning (Tao et al., 25 Sep 2025).

5. Empirical results and evaluation protocol

The headline result is that MOSS-ChatV achieves 87.2\% accuracy on the MOSS-Video test set. The framework also improves performance on general video reasoning benchmarks, including MVBench and MMVU, and the gains are reported as consistent across different architectures, including Qwen2.5-VL and Phi-2 (Tao et al., 25 Sep 2025).

Selected benchmark figures reported in the summary are as follows:

Benchmark	Baseline(s)	MOSS-ChatV
MOSS-Video Test	Qwen2.5-VL 67.0; Video-R1 73.3	86.6/87.2
MVBench	Qwen2.5-VL 67.1; Video-R1 63.9	67.6
MMVU	Qwen2.5-VL 60.0; Video-R1 64.8	66.2
RTV-Bench	Qwen2.5-VL 32.8; Video-R1 46.5	46.6

The reported ablations attribute a central role to the process reward. RL without process reward leads to unstable or trivial reasoning, whereas RL with process reasoning reward stabilizes output length and quality. The summary also states that supervised fine-tuning alone does not match the gains from RL+PRR. In addition, improvements are described as robust across varying frame counts, and benefits are reported not only for Qwen2.5-VL (Qwen2.5-7B) but also for TinyLLaVA-3B (Phi-2) and different vision encoders such as SigLIP (Tao et al., 25 Sep 2025).

Reasoning quality is further assessed with GPT-4o-as-judge, using four dimensions: Reasoning-Answer Consistency (0 or 1), Repetitiveness (0–10), Logical Coherence & Knowledge Accuracy (0–10), and Video Content Relevance (0–10). On this evaluation, MOSS-ChatV is reported to produce more consistent and stable reasoning traces, and to score higher than baselines on all these dimensions (Tao et al., 25 Sep 2025).

6. Scope, interpretation, and naming context

MOSS-ChatV is best understood as a framework for temporally grounded process optimization in multimodal reasoning. Its contribution is not limited to higher answer accuracy; it is also directed at the internal quality of reasoning traces. This emphasis distinguishes systems that merely solve a video question from systems whose reasoning remains aligned with object-state transitions and causal-temporal structure in the scene (Tao et al., 25 Sep 2025).

The framework’s rule-based reward design is also notable. Because the process reward depends on annotated reasoning traces rather than on a learned auxiliary reward model, the method is described as efficient and broadly applicable across architectures. This suggests a deliberate trade-off: rather than learning a separate judge, MOSS-ChatV operationalizes alignment directly through SDTW over segmented reasoning steps (Tao et al., 25 Sep 2025).

The acronym MOSS is used in multiple unrelated research contexts. In dialogue systems, for example, “MOSS: End-to-End Dialog System Framework with Modular Supervision” refers to an encoder-decoder framework that incorporates supervision from natural language understanding, dialog state tracking, dialog policy learning, and natural language generation (Liang et al., 2019). MOSS-ChatV instead belongs to the literature on multimodal LLMs, reinforcement learning, and video temporal reasoning.

Within that scope, MOSS-ChatV represents a shift from answer-only supervision toward reasoning-process supervision. A plausible implication is that such process-level objectives may become increasingly important wherever multimodal systems are expected not only to answer correctly, but to expose reasoning that remains faithful to the temporal evidence from which the answer is derived.