PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models

Abstract: Physical principles are fundamental to realistic visual simulation, but remain a significant oversight in transformer-based video generation. This gap highlights a critical limitation in rendering rigid body motion, a core tenet of classical mechanics. While computer graphics and physics-based simulators can easily model such collisions using Newton formulas, modern pretrain-finetune paradigms discard the concept of object rigidity during pixel-level global denoising. Even perfectly correct mathematical constraints are treated as suboptimal solutions (i.e., conditions) during model optimization in post-training, fundamentally limiting the physical realism of generated videos. Motivated by these considerations, we introduce, for the first time, a physics-aware reinforcement learning paradigm for video generation models that enforces physical collision rules directly in high-dimensional spaces, ensuring the physics knowledge is strictly applied rather than treated as conditions. Subsequently, we extend this paradigm to a unified framework, termed Mimicry-Discovery Cycle (MDcycle), which allows substantial fine-tuning while fully preserving the model's ability to leverage physics-grounded feedback. To validate our approach, we construct new benchmark PhysRVGBench and perform extensive qualitative and quantitative experiments to thoroughly assess its effectiveness.

• The paper introduces a unified reinforcement learning framework that enforces rigid body dynamics in video generative models.
• It employs a novel Physics-Grounded Reward metric combining trajectory offset and collision detection to enhance physical realism.
• Experimental results show superior performance in IoU and trajectory fidelity, setting a new benchmark for physics-aware video synthesis.

PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models

Introduction

Physically plausible video synthesis remains an unsolved problem in generative modeling. While large diffusion transformer (DiT) architectures have greatly advanced visual fidelity and semantic diversity in video generation, their outputs often exhibit limited physical realism, especially regarding rigid body motion governed by Newtonian mechanics. The neglect of explicit physical priors in pretrain–finetune training paradigms leads to a persistent gap between visual plausibility and physical correctness.

"PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models" (2601.11087) systematically addresses this challenge. The paper introduces a unified reinforcement learning (RL) framework that directly enforces rigid body dynamics in generative models, focusing on collision events and trajectory fidelity. More specifically, the proposed Mimicry-Discovery Cycle (MDcycle) is a novel post-training schema integrating data-driven imitation and physics-aware exploration, enhancing consistency with physical laws while maintaining synthesis quality.

Motivation and Problem Statement

Standard DiT-based video generation leverages flow matching or diffusion objectives in the latent space, emphasizing pixel-level accuracy and distributional alignment. However, such models disregard spatio-temporal physical constraints—discarding object rigidity and lawful motion in favor of global denoising. Even when physical priors are introduced as auxiliary objectives, model optimization relegates them to suboptimal conditions, leading to unstable learning of rigid motion and frequent physical artifacts (Figure 1).

Figure 1: The core idea of PhysRVG, contrasting conventional distribution-focused loss with the proposed physics-grounded reward, which enables the persistent retention of physical knowledge in model parameters.

RL approaches with subjective or perceptual feedback also have limitations, as they generally infer physical plausibility only indirectly and lack stable, fine-grained physical supervision. Hence, there is a need for a feedback mechanism that evaluates physical fidelity quantitatively—particularly for events such as collisions and canonical trajectories of rigid bodies.

Methodology

Physics-Grounded Reward Modeling

The core innovation of PhysRVG lies in its construction of a Physics-Grounded Metric for RL rewards. Given initial video frames and a text prompt, video generation is formulated as a future synthesis task. Object centers are manually annotated in the first frame, and subsequent positions are propagated using SAM2 segmentation. The metric consists of two components:

• Trajectory Offset (TO): Quantifies the mean deviation between generated and ground-truth trajectories, computed as the frame-wise Euclidean distance between corresponding object centers.
• Collision Detection: Identifies frames where an abrupt acceleration change occurs (via Newton’s second law), marking high-impact collision events. Temporal weights upweight reward penalties during these frames to ensure precise modeling of rigid body interactions.

The final reward is the temporally weighted negative TO, guiding policy updates toward minimizing trajectory discrepancies with an explicit bias towards accurate collision modeling (Figure 2).

Figure 2: The PhysRVG framework extracts motion masks, computes trajectory offset, performs collision detection, and synthesizes a temporally weighted reward signal for RL-based optimization.

The entire reward pipeline is differentiable owing to integration with segmentation and trajectory extraction, supporting large-scale, sample-efficient RL training.

Mimicry-Discovery Cycle (MDcycle)

To address RL instability (large gradient variance, poor exploration in sparse-reward regimes, and slow convergence), the MDcycle framework alternates between two stages:

• Mimicry: On challenging samples (high mean trajectory offset), the model optimizes pixel-level Flow Matching loss, effectively distilling perceptual and contextual information.
• Discovery: On easier samples, RL is prioritized using the physics-grounded reward, progressively internalizing collision and trajectory laws.

The cycle adaptively transitions between objectives based on batch-level performance, stabilizing optimization and promoting efficient physical knowledge acquisition.

Figure 3: (a) RL loss on samples with varying quality. (b) Dynamics of sample assignment to Mimicry and Discovery branches, illustrating adaptive task balancing during training.

RL Algorithmic Backbone

PhysRVG employs Group Relative Policy Optimization (GRPO), which uses groupwise advantage estimation and importance weighting, further extended with a hybrid SDE–ODE sampling protocol for efficient exploration in high-noise regions. All transformer blocks undergo full-parameter fine-tuning in early stages, with LoRA-style parameter-efficient tuning for physics-aware post-training.

Benchmark and Evaluation

The paper introduces a curated benchmark for rigid body video dynamics, containing 700 videos annotated for collision, free fall, rolling, and pendulum scenarios. Both qualitative and quantitative metrics are employed—subject and background consistency, motion smoothness, dynamic degree, IoU, and TO.

Against state-of-the-art models (e.g., Wan2.2, Kling2.5, Magi-1), PhysRVG achieves superior IoU and the lowest TO, substantiating its claim of enhanced physical realism without sacrificing perceptual quality. V2V models generally outperform I2V counterparts due to richer context, with PhysRVG further boosting rigid body consistency.

Figure 4: Representative qualitative comparisons indicate competing models' failures (static objects, trajectory errors, or object distortion), while PhysRVG outputs physically consistent, temporally coherent rigid body motions.

Ablation studies confirm the necessity of collision weighting, the importance of MDcycle versus vanilla RL or fine-tuning alone, and the robustness of performance to reward hyperparameters.

Figure 5: Ablation study highlighting the effect of collision detection on the reward signal and resulting generation fidelity.

Figure 6: (a) Reward curve stability across RL and MDcycle. (b)-(d) Distributional metrics for training strategies, verifying MDcycle's efficiency and quality contribution.

Theoretical and Practical Implications

PhysRVG is the first video generative model to enforce physical laws as hard constraints in high-dimensional pixel space using RL, moving beyond data augmentation or simulator-guided conditioning. This approach makes physical knowledge a core optimization target, not peripheral guidance. The integration with flow-model RL schemes (Flow-GRPO, SDE exploration) provides a scalable framework suitable for both foundation and task-specific video models.

Practically, PhysRVG has direct implications for physically-grounded video editing, robotics perception, scientific simulation, and style-to-motion transfer tasks where physical realism is paramount. The framework’s adaptability to new reward metrics and motion priors opens avenues for learning broader classes of dynamic priors (articulated body, nonrigid, or continuum mechanics).

Limitations and Future Directions

While addressing rigid body realism, the reward remains trajectory-focused and does not penalize nonphysical changes unrelated to motion (e.g., color distortion, appearance of extraneous objects). Multi-scale or more comprehensive reward design remains challenging. Full-parameter RL remains unstable in the high-dimensional video domain; thus, advances in sample-efficient, large-batch RL and adaptive reward shaping are needed.

An automatic mechanism for dynamic threshold tuning in MDcycle could further automate the balance between exploration and exploitation. As generative physical knowledge advances, strong considerations for adversarial misuse, content provenance, and synthetic content detection are necessary.

Conclusion

PhysRVG provides a robust, physics-aware RL framework for training video generative models, achieving state-of-the-art physical realism in rigid body scenarios while preserving perceptual quality. Through explicit reward design and adaptive training schema, it demonstrates for the first time that transformer-based models can internalize and respect the constraints of classical mechanics. Future work will expand the approach to more complex physical regimes and develop holistic evaluation protocols for comprehensive generative physical intelligence.

Figure 7: Evaluation metric design, summarizing the computation of IoU and trajectory offset (TO) as primary measures for assessing rigid body consistency.