VQAThinker: Reasoning-Based Video Quality Assessment

• VQAThinker is a reasoning-based framework for no-reference video quality assessment that integrates quality scoring with interpretable diagnostic explanations.
• It combines a large multimodal model with reinforcement learning via GRPO and tailored rewards to boost OOD generalization and temporal consistency.
• The framework aligns score prediction with perceptual reasoning, achieving state-of-the-art performance across diverse benchmark datasets.

VQAThinker is a reasoning-based framework for no-reference Video Quality Assessment (VQA) that uses a Large Multimodal Model (LMM) and reinforcement learning to jointly perform video quality understanding and quality scoring. It is designed to address two limitations identified for existing VQA models—poor generalization to out-of-distribution (OOD) videos and limited explainability—by generating both a reasoning trace and a scalar quality score under score-level supervision. The framework is trained with Group Relative Policy Optimization (GRPO) and three VQA-specific rewards: a bell-shaped regression reward, a pairwise ranking reward, and a temporal consistency reward (Cao et al., 8 Aug 2025).

1. Conceptual scope and problem setting

VQAThinker addresses perceptual video quality assessment rather than question answering over video content. Its target is the objective quantification of perceptual quality degradation in alignment with human visual perception, while also producing interpretable diagnostic output. In the formulation described for the framework, the model receives a video $\bm{v}$ together with a textual prompt $p$ and produces a quality-aware response $q$ that includes a reasoning trace inside > ... and a scalar quality score inside <answer> ... </answer> (Cao et al., 8 Aug 2025).

The motivating claim is that standard supervised fine-tuning tends to overfit and generalize poorly, especially under distribution shift. VQAThinker therefore adopts a reasoning-centric reinforcement learning formulation in which scoring and explanation are optimized together. This suggests a shift from treating VQA as pure regression toward treating it as a structured perceptual decision process.

2. Model architecture and temporal representation

The framework uses an LMM backbone, described with examples such as InternVL3-8B, and augments it with an explicit temporal pathway. A frozen motion encoder, exemplified by SlowFast, extracts local temporal dynamics, and a motion projector maps those temporal features into the language space so that the LMM can reason jointly over spatial and temporal evidence (Cao et al., 8 Aug 2025).

The model inference is written as

$$q = \mathcal{Q}_\theta(p, \bm{v}, \mathcal{M}(\bm{v}))$$

where $\mathcal{Q}$ denotes the LMM with the motion projector and $\mathcal{M}$ denotes the motion feature extractor. In this arrangement, the motion extractor is kept frozen while the remaining components are optimized through GRPO. The stated purpose of this design is to make temporal evidence directly available to the LLM, rather than relying only on frame-level spatial representations.

This architectural choice is significant because video quality depends not only on frame content but also on temporal stability, motion coherence, and temporally distributed artifacts. A plausible implication is that the frozen motion branch provides a stable inductive bias for temporal quality cues, while the LMM handles explanation and final score prediction.

3. Reinforcement learning formulation and reward design

VQAThinker uses GRPO as its training algorithm. For each training video, multiple responses are sampled under the old policy, each response is assigned a reward, and the response reward is standardized within the group. The standardized advantage is defined as

$$a_k(\bm{v}_i) = \frac{r_k(\bm{v}_i) - \mu(r(\bm{v}_i))}{\sigma(r(\bm{v}_i))}$$

and the clipped GRPO objective is given as

$$\mathcal{J}(\theta) = \mathbb{E} \left[ \min(\rho_i a_k, \bar{\rho}_i a_k) - \beta \cdot D_{\text{KL}}(\pi_\theta(q_k|\bm{v}_i)\|\pi_{\text{ref}}(q_k|\bm{v}_i)) \right]$$

with $\rho_i$ the policy ratio and $\bar{\rho}_i$ its clipped version (Cao et al., 8 Aug 2025).

The distinctive part of VQAThinker is its VQA-specific reward suite.

Reward	Definition in the framework	Intended role
Bell-shaped regression reward	Increases rapidly as the prediction error decreases and becomes progressively less sensitive near the ground truth	Precise score regression
Pairwise ranking reward	Guides the model to correctly determine the relative quality between video pairs	Relative quality ordering
Temporal consistency reward	Encourages the model to prefer temporally coherent videos over their perturbed counterparts	Temporal robustness
Format reward	Encourages generation in the required <think>...<answer>... format	Explainability and validity

The bell-shaped regression reward is written as

$p$0

where $p$1 is the predicted score, $p$2 is the ground-truth score, $p$3 is a scaling factor, and $p$4 is a sensitivity parameter. The temporal consistency reward adds a bonus when the original video is preferred over a temporally perturbed counterpart and the average reward crosses a confidence threshold:

$p$5

The total reward is

$p$6

The reward design encodes four distinct desiderata: score fidelity, ordinal consistency, temporal sensitivity, and interpretable output format.

4. Empirical evaluation and benchmark performance

The scoring experiments are reported on standard user-generated-content VQA datasets, including LSVQ test, LSVQ 1080p, KoNViD-1k, LIVE-VQC, and YouTube-UGC, together with OOD datasets including LIVE-YT-Gaming, CGVDS, HFR, Waterloo-IVC-4K, and VDPVE. On these in-domain and OOD benchmarks, VQAThinker is reported to achieve state-of-the-art performance, and on the difficult OOD datasets it is said to show a 19.1% relative gain in average SRCC over prior RL-based and supervised models (Cao et al., 8 Aug 2025).

The framework is also evaluated on video quality understanding tasks. For distortion attribution, the reported benchmark is FineVD, where VQAThinker is described as surpassing all LMMs and explainable VQA models, including models explicitly trained on distortion-labeled data. For quality description, the benchmark is Q-Bench-Video, where VQAThinker is described as nearly matching the highest-performing instruction-tuned model despite being trained without instruction annotation. These results are presented as evidence that the generated reasoning traces are not merely format-compliant text, but are useful for diagnostic quality analysis.

The reported empirical picture therefore has two parts. First, the framework improves rank-based and correlation-based quality scoring under distribution shift. Second, it improves the semantic quality of explanations used for distortion attribution and descriptive assessment. This combination is central to the framework’s identity.

5. Explainability, ablations, and training dynamics

The framework’s explainability is grounded in its output protocol: every response includes a reasoning trace and a final score. The examples described for the model indicate that the trace explicitly mentions dominant distortions such as blur, artifacts, and temporal instability. This is presented as diagnostic explainability rather than post hoc visualization or auxiliary saliency output (Cao et al., 8 Aug 2025).

The ablation findings emphasize that the reported gains depend on the interaction of multiple components. Removing the bell-shaped reward causes a significant performance drop near ground truth; omitting the ranking or temporal rewards reduces OOD and generalization performance; excluding the motion encoder or the reasoning-trace generation degrades both in-domain and OOD results; and replacing GRPO with supervised fine-tuning lowers performance and robustness. The authors therefore attribute the framework’s behavior not to a single scoring trick, but to the combined effect of reward shaping, temporal representation, and reasoning-based generation.

A plausible implication is that VQAThinker treats explanation as part of the optimization target rather than as an auxiliary output. In that respect, its training objective aligns score prediction with interpretable perceptual reasoning rather than separating them into independent modules.

6. Relation to adjacent work and terminological clarifications

A plausible source of confusion is that “VQA” in VQAThinker denotes Video Quality Assessment, not visual question answering. This distinction matters because the framework is not designed to answer semantic questions about video content, but to estimate perceptual quality and explain the factors behind that estimate. Related quality-assessment work such as Q-Ponder similarly unifies score regression and reasoning description, but it is formulated for visual quality assessment in images through a two-stage cold-start plus GRPO pipeline, whereas VQAThinker is centered on video, temporal dynamics, pairwise ranking across videos, and temporal consistency rewards (Cai et al., 3 Jun 2025).

It is also distinct from VQQA, whose title expands to “Video Quality Question Answering.” VQQA uses a multi-agent framework to evaluate generated videos and iteratively refine prompts through semantic gradients in a black-box optimization loop, while VQAThinker is a no-reference perceptual scoring framework trained under score-level supervision (Song et al., 12 Mar 2026). The terminological overlap therefore masks a methodological difference: one system optimizes video generation prompts, and the other estimates perceptual video quality.

Within the broader trajectory of multimodal reasoning research, VQAThinker illustrates a specific claim: reinforcement learning can be used not only to elicit chain-of-thought-style outputs, but also to improve OOD generalization in perceptual assessment tasks when the rewards are tailored to domain structure. In this case, that structure is defined by precise regression behavior, relative ranking, temporal coherence, and interpretable explanation (Cao et al., 8 Aug 2025).