VideoRewardBench: Video Reward Evaluation

• VideoRewardBench is a comprehensive benchmark that evaluates multimodal reward models on video understanding across four key dimensions: perception, knowledge, reasoning, and safety.
• It standardizes evaluation via preference judgments on video-text prompt triplets, measuring performance with metrics like overall and macro average accuracy.
• The benchmark aggregates 1,559 preference pairs from diverse video datasets to compare generative, discriminative, and semi-scalar models, setting a robust baseline in the field.

Searching arXiv for the exact benchmark and closely related follow-up work. VideoRewardBench is a benchmark for evaluating multimodal reward models (MRMs) on video understanding. It was introduced as the first comprehensive benchmark spanning four core aspects of video understanding—perception, knowledge, reasoning, and safety—and it formalizes evaluation as a preference judgment problem over triplets $(x, y_c, y_r)$, where $x$ is a video-text prompt, $y_c$ is a chosen response, and $y_r$ is a rejected response (Zhang et al., 30 Aug 2025). In this formulation, the benchmark does not test open-ended answer generation directly; rather, it tests whether a reward model can reliably identify the better of two candidate responses to a video-grounded query. The benchmark is designed to address limitations that earlier video-domain reward evaluations exhibited in question diversity, evaluation dimensions, and coverage of different reward-model architectures (Zhang et al., 30 Aug 2025).

1. Scope and problem formulation

VideoRewardBench was proposed against a background in which video-domain reward-model evaluation was described as incomplete in three specific respects: prior benchmarks had a limited number and diversity of questions, focused mainly on perception, and inadequately covered diverse types of MRMs (Zhang et al., 30 Aug 2025). Its defining intervention is therefore simultaneously taxonomic and methodological. Taxonomically, it expands evaluation from perception-only settings to perception, knowledge, reasoning, and safety. Methodologically, it evaluates three MRM paradigms—generative, discriminative, and semi-scalar—within a single benchmark (Zhang et al., 30 Aug 2025).

Perception is further divided into long-form perception and short-form perception, so the benchmark effectively reports macro average accuracy over five evaluation areas: long-form perception, short-form perception, knowledge, reasoning, and safety (Zhang et al., 30 Aug 2025). This separation is consequential because the benchmark is intended to probe both short-horizon visual discrimination and longer-context video comprehension.

Each benchmark item is a preference triplet. For generative MRMs, evaluation uses pairwise ranking: the model sees the video-question prompt together with two candidate responses and must choose the better one. For discriminative and semi-scalar MRMs, evaluation uses pointwise scoring: each response is scored independently, and the prediction is counted as correct when the chosen response receives the higher score, i.e. when

$s(x,y_c) > s(x,y_r).$

The reported metrics are per-dimension accuracy, overall accuracy, and macro average accuracy (Zhang et al., 30 Aug 2025).

2. Dataset composition and coverage

In the original benchmark report, VideoRewardBench contains 1,563 preference pairs, 1,482 unique videos, and 1,559 distinct prompts (Zhang et al., 30 Aug 2025). The benchmark is approximately balanced across its five major areas.

Area	Samples
Long-form perception	283
Short-form perception	413
Knowledge	238
Reasoning	278
Safety	351

The benchmark is assembled from 10 source video datasets. Long-form perception is sourced from VCGBench-Diverse. Short-form perception draws from MVBench and VideoHallucer. Knowledge draws from MMWorld, MMVU, and Video-MMMU. Reasoning draws from Video-MME, MMBench-Video, and VSI-Bench. Safety draws from Video-SafetyBench (Zhang et al., 30 Aug 2025).

The corpus spans multiple video durations: 888 short videos with duration $\leq 1$ minute, 492 medium videos of 1–5 minutes, and 102 long videos of more than 5 minutes (Zhang et al., 30 Aug 2025). The benchmark report also gives linguistic summary statistics: the average question word count is 28.8, the average response word count is 103.8, and the chosen and rejected responses have average word counts of 102.9 and 104.6, respectively (Zhang et al., 30 Aug 2025). This is used in the benchmark analysis to argue that preference labels are not simply explained by response length.

The scale claim is central to the benchmark’s positioning. VideoRewardBench is described as containing 1,559 distinct prompts, more than 15 times the number found in the most question-rich prior benchmark (Zhang et al., 30 Aug 2025). This emphasis on prompt diversity distinguishes it from earlier video reward evaluations that were summarized as having very few prompts and a narrow task profile.

3. AI-assisted construction and annotation pipeline

VideoRewardBench is built with a three-stage AI-assisted data pipeline: prompt collection, multi-stage filtering, and response collection with preference annotation (Zhang et al., 30 Aug 2025). Prompts are drawn from the source benchmarks with selection choices intended to increase difficulty and coverage.

The filtering stage is dimension-specific. For short-form perception, knowledge, and reasoning, the pipeline discards prompts whose videos are longer than 10 minutes, asks a strong model to answer the question without video, removes prompts that it answers correctly, and then feeds the full video-text prompt to Qwen2-VL-7B-Instruct and removes prompts it also answers correctly (Zhang et al., 30 Aug 2025). This procedure is intended to eliminate prompts answerable without video and to retain tasks that are neither trivial nor degenerate. For long-form perception and safety, the pipeline mainly filters out videos longer than 10 minutes (Zhang et al., 30 Aug 2025).

Response collection and annotation also vary by dimension. For long-form perception, responses are sampled from LLaVA-Video-72B, Qwen2.5-VL-72B, and three proprietary models; three models are randomly selected per prompt to generate responses, these responses are paired into three preference pairs, and three human annotators judge which response is better or whether the pair is a tie and assign preference strength. Majority vote determines the final label, and pairs with no consensus or ties are discarded (Zhang et al., 30 Aug 2025).

For short-form perception, the benchmark uses ground-truth answers directly: the correct answer becomes the chosen response, an incorrect option or the opposite yes/no answer becomes the rejected response, and no human annotation is used (Zhang et al., 30 Aug 2025). For knowledge and reasoning, responses are sampled from three leading proprietary models; one model generates 10 responses per prompt, prompts are removed if all 10 are uniformly right or wrong, and a preference pair is formed from one response with the correct final answer and one with the incorrect final answer. Three annotators then inspect the reasoning chain behind the correct response, and any pair with a critical flaw identified by any annotator is discarded (Zhang et al., 30 Aug 2025).

Safety uses a separate procedure. Six models are used, including the five from long-form perception plus one additional strong safety model. Each response is scored with RJScore from Video-SafetyBench and assigned to either successfully attacked or unsuccessfully attacked. Prompts are removed if all six responses fall into the same category, and only prompts with attack success rate greater than 50% are kept. A chosen/rejected pair is formed from one successfully attacked response and one unsuccessfully attacked response with similar length, and three annotators label the pair; only pairs where the chosen response is safe and the rejected response is attacked are retained (Zhang et al., 30 Aug 2025).

4. Evaluation protocol and model families

VideoRewardBench evaluates 28 MRMs across three categories: generative, discriminative, and semi-scalar (Zhang et al., 30 Aug 2025). The generative category is subdivided into proprietary models without critic training, open-source models without critic training, fast-thinking generative MRMs with critic training, and slow-thinking generative MRMs with critic training.

The proprietary generative models without critic training are GPT-4o-mini, GPT-4o, Claude-3.7-Sonnet, Gemini-2.5-flash, and Gemini-2.5-Pro. The open-source generative models without critic training are Aria, MiniCPM-o-2.6, mPLUG-Owl3-7B, Phi-3.5-Vision, InternVideo2.5-8B, InternVL3-8B, InternVL3-78B, LLaVA-OneVision-7B, LLaVA-OneVision-72B, LLaVA-Video-7B, LLaVA-Video-72B, Qwen2-VL-72B, Qwen2.5-VL-7B, and Qwen2.5-VL-72B. The critic-trained generative models include LLaVA-Critic-7B, LLaVA-Critic-72B, UnifiedReward, UnifiedReward-Think, R1-Reward, and Flex-Judge. The discriminative MRMs are IXC-2.5-Reward and Skywork-VL Reward, and the semi-scalar MRM is MM-RLHF-Reward (Zhang et al., 30 Aug 2025).

The protocol differs by model family. Generative MRMs are evaluated with pairwise ranking, with response order randomized to reduce position bias. Non-critic-trained generative models are prompted in a judge-style format similar to RewardBench, while critic-trained generative models use their official pairwise ranking prompts and inference code. Discriminative and semi-scalar MRMs instead score the split pairs $(x,y_c)$ and $(x,y_r)$ independently, and correctness requires the chosen response to score higher (Zhang et al., 30 Aug 2025).

For generative and semi-scalar models, the paper reports evaluation with temperature set to 0 and maximum output length 2048 (Zhang et al., 30 Aug 2025). The benchmark thus serves not merely as a dataset but as a protocolized cross-family evaluation suite in which fundamentally different reward-model interfaces are normalized to a common preference-comparison task.

5. Quantitative results and benchmark diagnostics

The benchmark report describes VideoRewardBench as challenging. The top overall accuracy is 63.6% for Gemini-2.5-Pro, followed by 63.2% for Claude-3.7-Sonnet and 63.0% for LLaVA-Critic-72B; GPT-4o reaches 57.0%, and Qwen2.5-VL-72B, the best open-source non-critic-trained model, reaches 53.3% (Zhang et al., 30 Aug 2025). Performance ranges from 63.6% at the top down to 34.6% for the weakest model, yielding nearly a 30-point spread (Zhang et al., 30 Aug 2025). Only four models exceed 60% overall: Gemini-2.5-Pro, Claude-3.7-Sonnet, LLaVA-Critic-72B, and Skywork-VL Reward (Zhang et al., 30 Aug 2025).

Several dimension-level trends are emphasized. Short-form perception is very hard for most models. Knowledge and reasoning are also difficult, especially for open-source models. Safety varies sharply by model family and instance: Claude-3.7-Sonnet reaches 82.9% on safety, while Qwen2.5-VL-72B drops to 44.7% (Zhang et al., 30 Aug 2025). These disparities are one reason the benchmark reports both overall and macro-average accuracy rather than a single undifferentiated number.

The analysis isolates three recurring findings. First, MRMs trained with reinforcement learning do not necessarily generalize better cross-modally to video. LLaVA-Critic-7B outperforms its base model by 1.9%, and LLaVA-Critic-72B outperforms its base model by 5.4%, but UnifiedReward-Think slightly drops relative to its base model, R1-Reward drops by 15.6% versus Qwen2.5-VL-7B, and Flex-Judge drops by 20% versus its base model (Zhang et al., 30 Aug 2025). This suggests that critic training and RL-style training are not interchangeable with respect to video transfer.

Second, inference-time scaling usually helps, except for discriminative MRMs. The paper samples $K \in [1,9]$ candidate judgments with temperature 1.0; generative MRMs are aggregated by majority vote, and semi-scalar MRMs are evaluated with both majority voting and score merging. Claude-3.7-Sonnet improves by 10.6% from $K=1$ to $x$0, Qwen2.5-VL-72B improves by 2% from $x$1 to $x$2, MM-RLHF-Reward improves from 54.5% to 56.5% with score merging, and R1-Reward gains 14.3% from $x$3 to $x$4 (Zhang et al., 30 Aug 2025). Discriminative MRMs do not benefit under this analysis because their outputs are deterministic scores.

Third, input frame count affects model families differently. The paper varies the number of input frames from 1 to 64. More frames often help, but not uniformly: critic-trained generative MRMs benefit the most, non-critic-trained generative MRMs improve less, semi-scalar MRMs are the least sensitive and may slightly decline, and discriminative MRMs fluctuate but stabilize with more frames. A concrete example is LLaVA-Critic-72B, which improves from 52.0% to 63.0% as frames increase from 1 to 64 (Zhang et al., 30 Aug 2025). By contrast, for Qwen2.5-VL, more frames can even hurt safety, with safety accuracy dropping by about 6% from 2 to 64 frames (Zhang et al., 30 Aug 2025).

6. Position in the benchmark landscape

VideoRewardBench occupies a specific place within the emerging literature on video reward modeling: it is the prior baseline benchmark for video-understanding reward evaluation against which later work explicitly measures progress (Wei et al., 8 May 2026). A later paper introducing Video Understanding Reward Bench (VURB) describes VideoRewardBench as having 1,559 preference samples, task-specific coverage over video understanding dimensions, only 35.9% of preference responses containing chain-of-thought-style reasoning, an average response length of 104 tokens, and a vanilla single-pass evaluation protocol that is vulnerable to position bias (Wei et al., 8 May 2026). In that framing, VideoRewardBench remains important as the established evaluation baseline, while VURB is presented as a more reasoning-aligned successor benchmark with long chain-of-thought reasoning in every preference response and majority voting evaluation (Wei et al., 8 May 2026).

The same later work reports that reward models trained on its new preference dataset, VideoDRM and VideoGRM, achieve state-of-the-art performance on both VURB and VideoRewardBench, reaching 64.7% and 64.6% overall accuracy on VideoRewardBench, respectively (Wei et al., 8 May 2026). This is significant because it places VideoRewardBench within an evolving benchmark lineage rather than treating it as an isolated artifact.

VideoRewardBench should also be distinguished from several adjacent but non-identical benchmark lines. MJ-BENCH-VIDEO is a large-scale video preference benchmark for video generation organized around Alignment, Safety, Fineness, Coherence & Consistency, and Bias & Fairness, rather than video understanding reward judgment per se (Tong et al., 3 Feb 2025). V2P-Bench evaluates video-language understanding with visual prompts for human-model interaction scenarios and uses multiple-choice accuracy rather than response-preference judgment (Zhao et al., 22 Mar 2025). VQPP is a benchmark for video query performance prediction in content-based video retrieval and is explicitly described as not being a reward benchmark in the RLHF sense, although it can provide reward-like signals for downstream optimization (Lutu et al., 19 Feb 2026). ExeVR-Bench, introduced in work on computer-use agents, evaluates reward modeling from execution video and instruction pairs rather than general video understanding responses (Song et al., 10 Mar 2026). These neighboring benchmarks clarify the specificity of VideoRewardBench: it is a benchmark for comparative evaluation of MRMs as judges of response quality in video understanding.