WR-Arena: World Reasoning Benchmark

• WR-Arena is a comprehensive benchmark for evaluating world models, defining high-level simulation, long-term planning, and counterfactual reasoning in embodied AI.
• It measures critical dimensions like action simulation fidelity, long-horizon forecasting, and simulative planning using rigorous, quantitative protocols.
• The suite integrates visuotactile inputs and cross-platform evaluations to bridge the sim-to-real gap and inform robust intelligent agent development.

WR-Arena (World Reasoning Arena) denotes a modern suite of benchmarks for evaluating world models (WMs) in both embodied intelligence and reasoning-driven simulation, unifying diverse tasks and quantitative protocols for assessing simulation fidelity, long-horizon simulation, and simulative reasoning. WR-Arena encompasses two principal lines: (1) embodied world-model evaluation with closed-loop visuotactile prediction and real-robot transfer, and (2) diagnostic world-model reasoning under multi-step instructions and counterfactual planning. It provides a rigorous testbed highlighting the disparity between synthetic perceptual metrics and the robust “internal simulator” capacity needed for general-purpose intelligent agents (Shang et al., 18 May 2026, Team et al., 26 Mar 2026).

1. Scope, Motivations, and Conceptual Goals

World models have become a foundational tool for both embodied AI and general reasoning agents. Whereas prior benchmarks have concentrated on short-term state prediction or pixel-level reconstruction, WR-Arena—the "World Reasoning Arena"—was introduced to audit the higher-level capabilities a true internal simulator must possess: prediction under high-level instructions, hypothetical reasoning over alternative futures, robustness to long temporal horizons, and reliable support for downstream planning or reinforcement learning (Team et al., 26 Mar 2026). WR-Arena’s motivation stems from deficiencies in earlier protocols, notably their neglect of:

• Semantically meaningful action simulation fidelity;
• Ability to sustain physically coherent trajectories across many rounds (long-horizon forecasting);
• Effective support for planning by simulating and comparing multiple candidate futures (simulative reasoning).

This diagnostic ambition is matched by the embodied-robotics variant, WorldArena 2.0 (Shang et al., 18 May 2026), which generalizes evaluation along sensory modality (adding visuotactile channels), task functionality (opening the loop to in-model RL policy optimization), and physical platform (covering both simulation and real-world robotics). Thus, "WR-Arena" has become a referent for benchmarks that stress both perception-driven and reasoning-driven metrics for world models.

2. Evaluation Dimensions and Formal Metrics

WR-Arena assesses models along three principal axes, each instantiated with rigorous, quantitative metrics and diverse data splits (Team et al., 26 Mar 2026):

• Action Simulation Fidelity: Measures whether the model can conditionally generate trajectories $\mathcal{R}(s_0, \mathcal{A})$ aligned to high-level instruction sequences, and whether diverse instructions from the same $s_0$ yield semantically distinct futures. Semantic alignment is scored by a vision–LLM as

$$\mathrm{Align}(s_0,\mathcal{A}) = \frac{1}{T}\sum_{t=1}^T \mathrm{Judge}(s_t | (s_{t-1},a_t)),$$

with counterfactual diversity as mean feature- or image-space distance across alternative rollouts. The aggregate Action Simulation Fidelity (ASF) is

$$\mathrm{ASF} = \lambda\,\mathrm{Align} + (1-\lambda)\,\mathrm{Div}, \quad \lambda\in[0,1].$$

• Long-horizon Forecast: Evaluates a model’s ability to propagate physically plausible state over many steps, using both video smoothness metrics based on optical flow (multi-round smoothness, $\mathrm{MRS}$) and generation consistency (exponential penalty for drift/degeneracy).
• Simulative Reasoning and Planning: Assesses how well a model supports a planner in generating, comparing, and selecting among candidate futures to optimize for explicit goals. Metrics include step-wise prediction accuracy, open-ended scenario success, and improvement over VLM-only planning.

These axes form a taxonomy (see Section 3) spanning agent-centric, environment-centric, structured, and open-ended domains and instructions.

3. Benchmark Task Taxonomy and Datasets

The WR-Arena suite comprises a wide array of test domains, systematically assembled to probe both breadth and reasoning depth (Team et al., 26 Mar 2026). Key features:

• Domain Diversity: Includes driving scenes, kitchen/cooking, household tasks, tabletop block manipulation, and robot arm environments.
• Instruction Complexity: Ranges from single-turn discrete actions to open-ended high-level plans and structured scene-level interventions.
• Data Regimes: For action simulation and long-horizon forecasting, curated sets of $\sim$5,000 initial world states (each with up to 5 instruction sequences) are used; for simulative planning, well-defined scenarios from WM-ABench, Agibot, and Language Table benchmarks serve as standard references.

Embodied WR-Arena (WorldArena 2.0) extends this taxonomy further, introducing visuotactile observation spaces and multiple physical testbeds:

• Simulators: RoboTwin 2.0 (bimanual, domain-randomized, 731 objects), LIBERO (Robosuite-based), and a real-world mobile manipulator (AgileX Split-Type ALOHA).
• Task Set: Contact-rich manipulation (e.g., "Insert HDMI," "Pour Water") and classic continuous control, with a focus on sim-to-real transfer and cross-platform consistency (Shang et al., 18 May 2026).

4. Methodological Protocols and Comparative Baselines

A standardized, reproducible protocol underpins every dimension of WR-Arena:

• Perceptual Evaluation: Models are compared using 16 video-quality metrics across six dimensions (visual, motion, content consistency, physics adherence, 3D, controllability), leveraging pre-specified rollout seeds per task and state-of-the-art vision-language judging models.
• Functional Evaluation: For each platform (simulated or real), two main protocols: (1) "data engine"—synthetic trajectory generation for policy training and real-world deployment; (2) "action planner"—closed-loop plan in the model, followed by real system execution.
• RL Utility: Policies are trained purely within the learned world-model environment and tested on the real system to obtain success rates; cumulative reward correlation (simulated vs. real) is calculated as $$\rho_R = \mathrm{corr}\left(\sum r_t^{sim}, \sum r_t^{real}\right)$$.

Comparative baselines assemble a representative frontier of world models, including:

• Commercial video-diffusion backbones (Veo 3.1, Wan 2.6),
• Embodied-specific models (Vidar, RoboScape, Ctrl-World, IRASim, WoW),
• Vanilla predictors (Cosmos-Predict, iVideoGPT).

In the multi-agent variant (Arena (Song et al., 2019)), the framework exposes agents to a Python–Unity interface, a configurable social tree for organizing group interactions and reward structures, and reference implementations for diverse multi-agent RL algorithms (D-PPO, SP, PB, MADDPG/MAAC, COMA).

5. Experimental Findings and Sim-to-Real Gaps

Quantitative results on WR-Arena and WorldArena 2.0 reveal both technological progress and critical limitations:

Model	Agent ASF	Env ASF	MRS	Gen Consist	Step-Wise (%)	Open-End Δ	Struct Δ
PAN	70.3	47.0	53.6	64.1	56.1	26.7%	23.4%
MiniMax	72.3	51.7	30.7	57.2	–	–	–
Gen-3	45.3	47.3	47.8	45.1	–	–	–
Cosmos2	58.0	44.0	17.3	59.2	31.1	0%	4.3%
WAN 2.2	65.0	39.3	20.5	37.6	–	–	–

• Action Simulation Fidelity: Models demonstrate a consistent agent-centric superiority (~11.5 points higher ASF) versus environment-centric interventions; no model surpasses 60% Env ASF, reflecting persistent limitations in scene-level control and counterfactual diversity.
• Long-horizon Forecasting: Error accumulation is severe; even leading models achieve ≤54% multi-round smoothness and ≤64% consistency, with sharp deterioration after 5–6 rounds.
• Simulative Planning: Only select models (e.g., PAN) reliably improve open-ended and structured planning success rates over VLM-only baselines (maximum increments ~26–23%).

In embodied settings (Shang et al., 18 May 2026), tactile-enhanced world models reach up to 100% success on some contact tasks (e.g., HDMI plug), but fail entirely on others (e.g., bottle lifting). Training RL policies in learned models often yields real-world success rates notably below simulator-trained policies (e.g., ~70% vs. ~87% on "Adjust Bottle"). There is strong metric correlation between simulators but poor transfer to real-robot performance; many models drop to <20% on real ALOHA tasks.

Crucially, perceptual metrics (e.g., PSNR/SSIM) for video predictions are only weakly correlated with meaningful task success in the real world, underscoring the need for physical evaluation.

6. Architectural and Modal Innovations

WR-Arena and related protocols prompt advances in world-model architecture. Notably:

• Visuotactile Models: Integration of vision (𝒱) and tactile (𝒯) inputs, via plug-in tactile VAEs and two-stream denoising networks, supports more robust contact-rich simulation. Prediction losses are jointly optimized:

$$\mathcal{L} = \mathcal{L}^{\mathrm{vision}} + \mathcal{L}^{\mathrm{tactile}}$$

allowing explicit balancing of sensory streams.

• Interactive RL-as-Environment: Learned models are formalized as POMDP approximators ($\hat{\mathcal P}_\phi$), enabling closed-loop policy training (via policy gradients) and deployment on physical hardware.
• Platform Generalization: Multi-platform benchmarks (simulator and real robot) with cross-embodiment evaluation protocols permit direct study of sim-to-real transfer errors and robustness.
• Event-Aware Generative Models: Recent developments involve event-aware bidirectional fusion (e.g., EA-WM (Yang et al., 7 May 2026)), spatial projection of action fields, and fusion of kinematic and visual latents for sharper physical interaction modeling.

7. Open Challenges and Prospective Directions

WR-Arena establishes clear targets and exposes specific gaps remaining in world model research:

• No current model approaches human-level performance (>95%) in instruction following, multi-step consistency, or hypothetical planning. Performance drops sharply with increased simulation length or real-world embodiment.
• Tactile and multimodal fusion yields dramatic gains in some tasks but leaves substantial gaps in others, indicating modality-specific bottlenecks.
• There is a persistent, large sim-to-real gap: models with high simulator correlation underperform drastically on real robotics platforms, necessitating richer sensory inputs, more diverse and physically-distributed datasets, and sharper reward functional approximations.
• Perceptual video-quality metrics fail to predict physical or planning task performance reliably; direct task-based and physical success metrics are indispensable for diagnosis and advancement.

Prospective research directions highlighted by the creators include expansion to further sensory domains (audio, force), enrichment of task and embodiment distributions, design of longer-horizon RL scenarios, and systematic bridging of the sim-to-real divide. WR-Arena is positioned as a diagnostic and developmental guidepost for robust, general-purpose world model research (Team et al., 26 Mar 2026, Shang et al., 18 May 2026).