WorldRewardBench: Video Reward Benchmark

• WorldRewardBench is a benchmark that evaluates reward models by comparing expert human judgments on AI-generated videos with dynamic world-state transitions.
• It comprises around 6,000 expert-annotated pair-wise comparisons over 1,432 videos from 11 diverse generators, ensuring nuanced evaluation beyond mere visual quality.
• The benchmark emphasizes temporal consistency, causality, and logical coherence, highlighting failures in dynamics that standard perceptual metrics often overlook.

Searching arXiv for the cited paper and closely related work on video-generation evaluation and reward modeling. WorldRewardBench is a preference benchmark for evaluating whether automatic or learnable reward models can recover expert human judgments over AI-generated videos that are intended to realize the same conditioning image and prompt. Introduced alongside WorldReasonBench in "WorldReasonBench: Human-Aligned Stress Testing of Video Generators as Future World-State Predictors" (Wu et al., 11 May 2026), it is designed to complement world-state prediction by supplying calibrated human preference data over videos whose quality depends not only on visual realism but also on physical, social, logical, and informational coherence across time. The benchmark contains approximately 6,000 expert-annotated pair-wise preferences over 1,432 unique videos, and it supports both pair-wise and point-wise reward-model evaluation (Wu et al., 11 May 2026).

1. Conceptual role within world-state evaluation

WorldRewardBench was introduced to close a gap in video-generation evaluation. WorldReasonBench asks whether a model, given an initial frame plus instruction, can roll the world forward into a physically and logically coherent future video; WorldRewardBench asks the complementary question of whether automatic or learnable reward models can recover expert human preferences over such videos (Wu et al., 11 May 2026). The distinction is important because the underlying failure mode is not merely poor rendering quality. Commercial video generators such as Seedance2.0, Veo3.1-Fast, Sora2, and Kling have improved visual realism rapidly, yet standard perceptual metrics including FID, FVD, and LPIPS fail to detect semantic and causal errors, exemplified by cases such as an apple accelerating upward.

The benchmark is therefore framed around a specific evaluation target: dynamic failures involving gravity, causality, and detail, rather than only visual polish. In that sense, WorldRewardBench functions as a calibration layer over world-state prediction. WorldReasonBench provides 436 curated test cases with structured ground-truth QA annotations spanning four reasoning dimensions and 22 subcategories, while WorldRewardBench samples a high-quality subset of that material and converts it into expert preference data suitable for judging and reward modeling (Wu et al., 11 May 2026).

A recurrent misconception in video evaluation is that visually convincing outputs are adequate proxies for correct world modeling. The reported results directly challenge that view: videos can look convincing while failing dynamics, causality, or information preservation. WorldRewardBench operationalizes this discrepancy by asking whether a reward signal can prefer the more world-consistent video even when both candidates are visually plausible.

2. Dataset composition and annotation pipeline

WorldRewardBench is built from 1,432 distinct AI-generated videos produced by 11 generators on 436 benchmark cases, with approximately 6,000 balanced pair-wise preference comparisons among videos from the same case after confidence-aware filtering (Wu et al., 11 May 2026). The generators are drawn from both closed-source and open-source systems: Seedance2.0, Veo3.1-Fast, Sora2-8s/12s, and Kling on the closed-source side, and LTX2.3, Wan2.2-14B, UniVideo, Hunyuan 1.5, Cosmos-Predict2.5, and LongCat-Video on the open-source side.

Component	Specification
Unique videos	1,432
Benchmark cases	436
Pair-wise comparisons	Approximately 6,000
Generator count	11
Annotators	15 trained annotators

The annotation workforce consisted of fifteen trained annotators with backgrounds in video-generation research and multimodal evaluation. Each annotator completed a one-hour training session on the task, rubric, and common failure modes. No personal or sensitive data were collected, and all annotators volunteered under an anonymized protocol (Wu et al., 11 May 2026).

This composition gives the benchmark two distinct properties. First, pair-wise comparisons are made within the same case, so preferences are conditioned on identical input intent rather than across unrelated prompts. Second, the annotator training and filtering stages are intended to produce preference labels that are not reducible to superficial aesthetics alone. A plausible implication is that WorldRewardBench is less a generic video-quality benchmark than a specialized human-alignment benchmark for future-state fidelity.

3. Scoring rubric, preference construction, and reliability

The annotation process combines point-wise scoring with derived pair-wise preferences. Each video receives ratings on three 1–5 scales: Reasoning Quality, Temporal Consistency, and Visual Aesthetics. These scores are aggregated as

$$S(v) = 0.4\,S_{\text{reasoning}} + 0.3\,S_{\text{continuity}} + 0.3\,S_{\text{aesthetics}}.$$

Within each case, videos are then rank-ordered by $S(v)$, and pair labels are derived from the resulting score differences (Wu et al., 11 May 2026).

The pair construction procedure includes several explicit controls. Pairs with $|\Delta S| < 0.1$ are labeled ties, with the special cases $A = B = \text{Good}$ or $A = B = \text{Bad}$ if both $S(v) \leq 2.0$. High-margin pairs are down-sampled to preserve difficult comparisons, and strict preferences are balanced between $A > B$ and $B > A$. The benchmark therefore does not merely mirror raw score ordering; it actively increases the relative weight of nontrivial comparisons.

Reliability control is also explicit. Video ratings with range $> 1$ when two raters are involved, or $> 2$ when three raters are involved, trigger re-annotation until at least four consistent scores are obtained. On final scores, inter-annotator reliability is reported as strong, with Krippendorff’s $S(v)$0 and $S(v)$1 (Wu et al., 11 May 2026).

This scoring design has a particular methodological consequence. Because Reasoning Quality carries weight $S(v)$2, while Temporal Consistency and Visual Aesthetics each carry weight $S(v)$3, the benchmark encodes an explicit value hierarchy rather than treating all perceptual dimensions symmetrically. That hierarchy matches the benchmark’s stated objective of prioritizing correct world transition over surface realism.

4. Evaluation protocols and formal objectives

WorldRewardBench supports two complementary protocols for evaluating reward models or judges. In the pair-wise protocol, the input is two videos, designated Model A and Model B, with the same conditioning image and prompt. The judge is asked, via a single natural-language prompt, which video better realizes the intended world transition, prioritizing reasoning correctness over content continuity and visual appeal. The output space is $S(v)$4 (Wu et al., 11 May 2026).

In the point-wise protocol, the input is a single video with its conditioning image and prompt. The judge independently rates the video on the same three 1–5 dimensions, producing a continuous score $S(v)$5 after linear rescaling. Preferences can then be induced by comparing $S(v)$6 and $S(v)$7 using the tie threshold $S(v)$8. The stated strength of the pair-wise protocol is high agreement with human preferences on close comparisons, whereas the stated strength of the point-wise protocol is calibrated per-video reward suitable for ranking and optimization.

The benchmark summary also specifies standard objectives for reward-model training. For pair-wise reward modeling, if predicted rewards $S(v)$9 correspond to videos satisfying preference $|\Delta S| < 0.1$0, one may use the logistic ranking loss

$|\Delta S| < 0.1$1

where

$|\Delta S| < 0.1$2

A margin-based alternative is

$|\Delta S| < 0.1$3

For point-wise regression, if scalar ground-truth scores $|\Delta S| < 0.1$4 are available per video, one may minimize

$|\Delta S| < 0.1$5

Evaluation uses preference accuracy, Spearman’s rank correlation $|\Delta S| < 0.1$6, and Kendall’s $|\Delta S| < 0.1$7. The summary gives

$|\Delta S| < 0.1$8

where $|\Delta S| < 0.1$9 is the rank difference for model $A = B = \text{Good}$0 across human versus predicted ranking, and

$A = B = \text{Good}$1

with $A = B = \text{Good}$2 concordant and $A = B = \text{Good}$3 discordant pairs (Wu et al., 11 May 2026).

5. Empirical findings across generators and automatic judges

The experimental findings combine generator evaluation on WorldReasonBench with reward-model alignment results on WorldRewardBench. For generator performance, the headline reasoning score is

$A = B = \text{Good}$4

with

$A = B = \text{Good}$5

Under this metric, closed-source generators score 32–40% and open-source generators 14–18%, yielding a near twofold gap. On the multi-dimensional quality score $A = B = \text{Good}$6, closed-source systems score 50–60 and open-source systems 21–31. The process-completeness ratio $A = B = \text{Good}$7 is reported as 0.54–0.63 for open-source systems versus 0.82–0.91 for closed-source systems, indicating that open-source failures concentrate on dynamic, specifically temporal and mechanistic, reasoning (Wu et al., 11 May 2026).

Difficulty is not uniform across reasoning dimensions. Logic Reasoning and Information-Based subcategories remain the hardest for both generators and judges, with the best closed-source $A = B = \text{Good}$8 at only approximately 32% and approximately 47%, respectively. World Knowledge and Human-Centric transitions are easier, with closed-source $A = B = \text{Good}$9 up to 44–58%.

For reward-model alignment on WorldRewardBench, five automatic judges are reported: GPT-5.4, Gemini-3.1-Flash, and Qwen3.5-9B/27B in both Instruct and Thinking variants. In pair-wise accuracy without ties, Qwen3.5-9B-Thinking leads at 74.35%, GPT-5.4 reaches 71.36%, and Gemini-3.1 reaches 62.99%. In point-wise induced accuracy, Qwen3.5-9B-Thinking is reported with $A = B = \text{Bad}$0 (Spearman), outperforming other setups. Across judges, Information-Based reasoning remains the bottleneck, with pair-wise agreement dropping to approximately 60% and point-wise performance to 0.4–0.5 (Wu et al., 11 May 2026).

These results support a specific interpretive distinction. Pair-wise judging is strongest when selecting between close candidates, whereas point-wise scoring provides smoother and more calibrated feedback for reward-model training. The benchmark therefore treats pair-wise and point-wise evaluation not as competing formulations but as functionally differentiated modes of supervision.

6. Methodological implications, misconceptions, and extensions

WorldRewardBench is intended for three immediate uses. First, it can be used for reward-model training by applying the approximately 6,000 labeled pairs under the logistic loss to fine-tune a transformer-based judge. Second, it can be used to evaluate new video generators by applying both pair-wise and point-wise protocols and measuring human-alignment with accuracy, $A = B = \text{Bad}$1, and $A = B = \text{Bad}$2. Third, it can be used for calibration by comparing an automatic judge against the expert Elo ranking before deployment (Wu et al., 11 May 2026).

The benchmark is also framed against two common misconceptions. One is that standard perceptual metrics are sufficient for evaluating future-state fidelity; the benchmark is explicitly motivated by the failure of FID, FVD, and LPIPS to detect semantic and causal errors. The other is that visual plausibility implies correct world reasoning; the reported experiments show a persistent gap between visual plausibility and world reasoning, including failures in dynamics, causality, and information preservation. A plausible implication is that reward-model development for video generation cannot be reduced to better aesthetic scoring alone.

The recommended extensions are concrete. The summary proposes expanding the taxonomy to counterfactual or multi-agent scenarios, releasing fine-tuning recipes for reward models trained end-to-end on the pair data, incorporating numerical ground truth such as trajectories and physics simulators for hybrid video+data benchmarks, and supporting non-English QA and image sources to broaden coverage (Wu et al., 11 May 2026). By coupling WorldReasonBench, WorldRewardBench, and the accompanying evaluation toolkit and scripts, the project is positioned as an effort to shift evaluation toward reward models and automatic judges that penalize flawed world-state transitions rather than merely rewarding polished appearance.