---
title: 'MOSS-ChatV: RL Framework for Video Reasoning'
url: https://www.emergentmind.com/topics/moss-chatv
type: topic
---

# MOSS-ChatV: RL Framework for Video Reasoning

MOSS-ChatV is a reinforcement learning framework for video temporal reasoning in multimodal large language models (MLLMs). It is designed to address **process inconsistency**, a failure mode in which intermediate reasoning drifts from video dynamics even when the final answer is correct, thereby undermining interpretability and robustness. The framework introduces a **Dynamic Time Warping (DTW)-based process reward** that aligns generated reasoning traces with temporally grounded references, and it is trained and evaluated with **MOSS-Video**, a benchmark containing annotated reasoning traces. Reported results include **87.2\% on MOSS-Video (test)**, together with improvements on general video benchmarks such as **MVBench** and **MMVU** [2509.21113].

## 1. Problem formulation and conceptual basis

Video reasoning in MLLMs requires models to move beyond static perception toward coherent understanding of temporal dynamics in complex scenes. Within this setting, MOSS-ChatV treats **dynamic state prediction** as a key measure of video reasoning. The core observation is that answer correctness alone is insufficient: a model may produce the correct endpoint while its step-by-step rationale remains temporally unfaithful to the underlying video. MOSS-ChatV is therefore centered on **process-level supervision**, not only outcome-level optimization [2509.21113].

The framework addresses a specific gap in prior reinforcement-learning approaches for multimodal reasoning. Existing systems commonly optimize final accuracy or depend on auxiliary reward models. MOSS-ChatV instead uses a **rule-based reward** that aligns reasoning traces with annotated references. This design is intended to provide efficient process supervision **without auxiliary reward models**, while preserving temporal grounding in the reasoning trajectory itself [2509.21113].

A common misconception is to equate correct final answers with correct video reasoning. MOSS-ChatV is explicitly motivated by the opposite observation: intermediate reasoning may be inconsistent with the depicted temporal process even when the answer is correct. In that sense, the framework reorients evaluation from terminal correctness toward the internal temporal structure of inference [2509.21113].

## 2. Reinforcement-learning framework

MOSS-ChatV uses **Group Relative Policy Optimization (GRPO)**, described as a scalable, sample-efficient RL algorithm for language models and associated in the paper with DeepSeek-R1 [2501.12948]. For each input, the policy $\pi_\theta$ samples $G$ responses, computes a scalar reward for each response, and standardizes the resulting advantages as

$$
A_i = \frac{\mathcal{R}_i - \mu}{\sigma},\quad
\mu = \operatorname{mean}(\{\mathcal{R}_i\}), \quad
\sigma = \operatorname{std}(\{\mathcal{R}_i\})
$$

The reward used in training is decomposed into three components: an **accuracy reward**, a **format reward**, and a **process reasoning reward**. The total reward is

$$
\mathcal{R}_i = \mathcal{R}_{\mathrm{acc},i} + \mathcal{R}_{\mathrm{fmt},i} + \mathcal{R}_{\mathrm{proc},i}
$$

The **accuracy reward** is $1$ if the final answer is correct and $0$ otherwise. The **format reward** is $1$ if the output matches the required schema, such as exposing `<think>...</think>` reasoning, and $0$ otherwise. The **process reasoning reward** is computed by aligning generated reasoning against reference reasoning [2509.21113].

This decomposition is important because it separates three distinct desiderata: terminal correctness, structural compliance, and temporally coherent intermediate reasoning. A plausible implication is that the framework is intended to discourage superficially correct but procedurally ungrounded outputs, while also stabilizing the format required for process-level evaluation.

## 3. DTW-based process reasoning reward

The distinctive component of MOSS-ChatV is its **DTW-based process reward**, implemented through a “split-align” procedure. Both generated reasoning and reference reasoning—the content inside `<think>`—are segmented into discrete reasoning steps, for example at sentence level using `nltk`. This yields two sequences:

$$
Seq_\text{gen} = \{g_1, ..., g_m\}
$$

$$
Seq_\text{ref} = \{r_1, ..., r_n\}
$$

A textual similarity matrix $\mathbf{D} \in \mathbb{R}^{m \times n}$ is then constructed with entries

$$
D_{j,k} = 1 - \text{ROUGE}_{\text{avg}}(g_j, r_k)
$$

where

$$
\text{ROUGE}_{\text{avg}}(g_j, r_k) = \tfrac13 [ \text{ROUGE-1} + \text{ROUGE-2} + \text{ROUGE-L} ]
$$

Rather than applying ordinary DTW in a way that penalizes any extra reasoning, the framework uses **Subsequence Dynamic Time Warping (SDTW)**. SDTW aligns the reference reasoning as a contiguous process embedded within the generated trace, thereby ignoring extra steps outside the alignment. The method also allows controlled “jumps,” described as tolerance for minor deviations or missing steps, to balance flexibility and strictness. If the minimal alignment cost is $D_\text{sdtw}$, then the process reward is

$$
\mathcal{R}_\text{proc} = \exp(-\alpha D_\text{sdtw})
$$

This construction serves several purposes at once. It rewards temporal fidelity by favoring reasoning that matches annotated steps in order; it tolerates additional valid exploration; and it is explicitly intended to avoid the reward hacking observed with naive DTW, where models can “short circuit” by generating minimal reasoning to minimize the DTW path. The paper characterizes the annotated traces as a **“minimal gold standard”**: extra correct reasoning is tolerated, but reward is not inflated by repetition [2509.21113].

## 4. MOSS-Video benchmark

MOSS-ChatV is coupled to **MOSS-Video**, a benchmark constructed from **ShareGPT4Video** and designed for process-level supervision in video reasoning. The dataset contains **11,654 training samples (w/ 1,218 unique videos)** and **2,836 test samples (w/ 479 unique videos)**. Each sample includes a video, a question about future or current object states, a ground-truth answer, and step-by-step reasoning annotations [2509.21113].

The annotations are built around object-state dynamics. The benchmark includes both **coarse-grained** annotations of the form `<object, state, timestamp>` and **fine-grained** annotations of the form `<object, change description, timestamp>`, extracted using multiple passes of GPT-4o prompt-based annotation. Two properties are emphasized in the description of the dataset: **temporal state prediction**, because queries always require dynamic state understanding, and **reasoning trace supervision**, because each answer is paired with annotated multi-step temporal explanations [2509.21113].

The benchmark is also part of the training protocol. The **training split is used to fine-tune MOSS-ChatV**, while the **held-out split is reserved for evaluation**. This makes MOSS-Video both a supervised source of temporally grounded reasoning traces and an evaluation framework for held-out generalization under process-level reinforcement learning [2509.21113].

## 5. Empirical results and evaluation protocol

The headline result is that **MOSS-ChatV achieves 87.2\% accuracy on the MOSS-Video test set**. The framework also improves performance on general video reasoning benchmarks, including **MVBench** and **MMVU**, and the gains are reported as consistent across different architectures, including **Qwen2.5-VL** and **Phi-2** [2509.21113].

Selected benchmark figures reported in the summary are as follows:

| Benchmark | Baseline(s) | MOSS-ChatV |
|---|---:|---:|
| MOSS-Video Test | Qwen2.5-VL 67.0; Video-R1 73.3 | 86.6/87.2 |
| MVBench | Qwen2.5-VL 67.1; Video-R1 63.9 | 67.6 |
| MMVU | Qwen2.5-VL 60.0; Video-R1 64.8 | 66.2 |
| RTV-Bench | Qwen2.5-VL 32.8; Video-R1 46.5 | 46.6 |

The reported ablations attribute a central role to the process reward. RL **without process reward** leads to unstable or trivial reasoning, whereas RL with process reasoning reward stabilizes output length and quality. The summary also states that **supervised fine-tuning alone does not match the gains from RL+PRR**. In addition, improvements are described as robust across varying frame counts, and benefits are reported not only for **Qwen2.5-VL (Qwen2.5-7B)** but also for **TinyLLaVA-3B (Phi-2)** and different vision encoders such as **SigLIP** [2509.21113].

Reasoning quality is further assessed with **GPT-4o-as-judge**, using four dimensions: **Reasoning-Answer Consistency** (0 or 1), **Repetitiveness** (0–10), **Logical Coherence & Knowledge Accuracy** (0–10), and **Video Content Relevance** (0–10). On this evaluation, MOSS-ChatV is reported to produce more **consistent and stable reasoning traces**, and to score higher than baselines on all these dimensions [2509.21113].

## 6. Scope, interpretation, and naming context

MOSS-ChatV is best understood as a framework for **temporally grounded process optimization** in multimodal reasoning. Its contribution is not limited to higher answer accuracy; it is also directed at the internal quality of reasoning traces. This emphasis distinguishes systems that merely solve a video question from systems whose reasoning remains aligned with object-state transitions and causal-temporal structure in the scene [2509.21113].

The framework’s rule-based reward design is also notable. Because the process reward depends on annotated reasoning traces rather than on a learned auxiliary reward model, the method is described as efficient and broadly applicable across architectures. This suggests a deliberate trade-off: rather than learning a separate judge, MOSS-ChatV operationalizes alignment directly through SDTW over segmented reasoning steps [2509.21113].

The acronym **MOSS** is used in multiple unrelated research contexts. In dialogue systems, for example, **“MOSS: End-to-End Dialog System Framework with Modular Supervision”** refers to an encoder-decoder framework that incorporates supervision from natural language understanding, dialog state tracking, dialog policy learning, and natural language generation [1909.05528]. MOSS-ChatV instead belongs to the literature on multimodal large language models, reinforcement learning, and video temporal reasoning.

Within that scope, MOSS-ChatV represents a shift from answer-only supervision toward **reasoning-process supervision**. A plausible implication is that such process-level objectives may become increasingly important wherever multimodal systems are expected not only to answer correctly, but to expose reasoning that remains faithful to the temporal evidence from which the answer is derived.

Source: https://www.emergentmind.com/topics/moss-chatv