What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards (2512.00425v1)

Abstract: Recent video diffusion models can synthesize visually compelling clips, yet often violate basic physical laws-objects float, accelerations drift, and collisions behave inconsistently-revealing a persistent gap between visual realism and physical realism. We propose $\texttt{NewtonRewards}$, the first physics-grounded post-training framework for video generation based on $\textit{verifiable rewards}$. Instead of relying on human or VLM feedback, $\texttt{NewtonRewards}$ extracts $\textit{measurable proxies}$ from generated videos using frozen utility models: optical flow serves as a proxy for velocity, while high-level appearance features serve as a proxy for mass. These proxies enable explicit enforcement of Newtonian structure through two complementary rewards: a Newtonian kinematic constraint enforcing constant-acceleration dynamics, and a mass conservation reward preventing trivial, degenerate solutions. We evaluate $\texttt{NewtonRewards}$ on five Newtonian Motion Primitives (free fall, horizontal/parabolic throw, and ramp sliding down/up) using our newly constructed large-scale benchmark, $\texttt{NewtonBench-60K}$. Across all primitives in visual and physics metrics, $\texttt{NewtonRewards}$ consistently improves physical plausibility, motion smoothness, and temporal coherence over prior post-training methods. It further maintains strong performance under out-of-distribution shifts in height, speed, and friction. Our results show that physics-grounded verifiable rewards offer a scalable path toward physics-aware video generation.

• The paper introduces a post-training framework that uses differentiable, physics-grounded rewards to enforce Newtonian kinematics in video generation.
• It leverages proxies like optical flow and feature embeddings to apply constant acceleration and mass conservation constraints, reducing velocity and acceleration errors by an average of 9.75%.
• The study demonstrates that combining kinematic and mass rewards prevents reward hacking, ensuring persistent object motion and smoother temporal trajectories across diverse motion primitives.

Physics-Grounded Constraints in Video Generation: Enforcing Newton’s Laws via Verifiable Rewards

Introduction

Contemporary video generation models, powered extensively by diffusion architectures, have attained remarkable visual fidelity but struggle with violations of basic physical plausibility: objects floating, acceleration drift, and non-causal motion are prevalent artifacts. This deficit impedes their utility in downstream applications, especially domains requiring grounded simulation (robotics, training world models, autonomous driving). The paper "What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards" (2512.00425) proposes a post-training framework that enforces physical realism in video generators by constructing explicit, verifiable rewards derived from measurable physical proxies, grounded entirely in Newtonian kinematics and conservation laws.

Framework Overview

The proposed post-training pipeline augments a pretrained video generation model (e.g., OpenSora v1.2) by introducing physics-based reward functions. Critically, physical dynamical variables are not directly observable in generated frames; instead, the authors operationalize proxies: optical flow (computed by a differentiable model $\Psi$) for velocity, and perceptual feature embeddings (via V-JEPA 2) for mass conservation. Explicit kinematic and appearance-based constraints are then imposed, enabling gradient-based fine-tuning of the generator toward physically coherent motion.

Figure 1: The physics-grounded post-training pipeline utilizes optical flow and visual features as proxies, constructing Newtonian kinematic and mass rewards to fine-tune video generative models.

NewtonBench-60K: Dataset and Benchmarks

To enable rigorous evaluation, the authors introduce NewtonBench-60K, a large-scale synthetic video benchmark encompassing five canonical Newtonian Motion Primitives (NMPs): free fall, horizontal throw, parabolic throw, ramp slide down, and ramp slide up. Each scenario is simulated with high fidelity using Kubric for orchestration, PyBullet for rigid-body dynamics, and Blender for rendering. Out-of-distribution (OOD) splits are constructed to evaluate model generalization to unseen physical configurations (extreme heights, velocities, friction perturbation).

Figure 2: Depiction of the five NMPs and associated force diagrams; right-side couplings illustrate representative constant-acceleration trajectories within the simulation corpus.

Reward Construction: Measurable Proxies and Physical Constraints

The core methodological innovation lies in leveraging proxies to instantiate verifiable, differentiable reward signals:

Kinematic Constraint (Constant Acceleration):

Discrete Newtonian motion in the image plane is enforced by penalizing deviations from the second-order finite difference relation:

$$\boldsymbol{\phi}_{t+1} - 2\,\boldsymbol{\phi}_t + \boldsymbol{\phi}_{t-1} \approx 0$$

where $\boldsymbol{\phi}_t$ is the predicted optical flow. This effectively operationalizes Newton’s Second Law across all NMPs, regardless of specific initial or boundary conditions.

Mass Conservation Constraint:

Per-frame feature embeddings $\mathbf{z}_t$ (from V-JEPA 2) serve as a proxy for visual mass consistency. The model penalizes temporal deviations in embedding space, maintaining object persistence and discouraging degenerate solutions that eliminate the moving entity to trivially minimize motion residuals.

Empirical Results: Quantitative and Qualitative Analysis

Experiments compare the physics-designed reward (Newtonian kinematic + mass constraints) against prior state-of-the-art post-training strategies (PISA optical flow, depth, segmentation-based alignment). Metrics comprise both appearance-based (L2, Chamfer, IoU) and physics-based (velocity RMSE, acceleration RMSE) evaluations.

Numerical Gains and Consistency:

Across all five NMPs and both in-distribution and OOD data splits, the framework yields consistent improvements in physical plausibility and temporal coherence (average +9.75% gain) versus SFT and PISA baselines. Acceleration RMSE and velocity RMSE reductions directly confirm superior adherence to Newtonian dynamics.

Figure 3: Relative performance improvements of physics-grounded post-training across all Newtonian primitives. Consistent positive gains contrast with unstable, scenario-dependent effects of earlier approaches.

Reward Hacking Mitigation:

Ablation demonstrates the necessity of combining both constraints. Optimizing the kinematic reward alone induces reward hacking—object velocities collapse toward zero, visual disappearance ensues, and apparent metric improvements are illusory. The mass conservation term prevents this failure mode by enforcing object persistence and non-trivial dynamics.

Figure 4: The left panel (with mass reward) preserves object motion and persistence; right panel (without mass reward) exhibits reward hacking where objects vanish.

Analysis of Kinematic Residuals:

Direct computation of mean discrete second-order residuals evidences minimal deviation from constant acceleration only for the proposed method, with all prior variants exhibiting substantial structured violations.

Figure 5: Quantitative comparison of horizontal and vertical residuals. The physics-grounded method achieves lowest deviation from constant acceleration.

Transfer to Real-World Video:

The protocol generalizes beyond simulation—fine-tuning purely in synthetic domains robustly improves generated dynamics for 361 natural free-fall videos from the PISA benchmark, confirming the utility of measurable proxies as domain-agnostic physical anchors.

Figure 6: Example real-world free-fall video. Models trained with synthetic verifiable rewards generalize effectively to physical camera footage and natural gravitational motion.

Qualitative Trajectory Fidelity:

Temporal rollouts under various NMPs show that only the physics-constrained generator produces smooth, friction-respecting, and gravity-consistent object paths. Baselines exhibit unnatural drift, jitter, or object disappearance.

Figure 7: For parabolic throws, vanilla SFT violates physics, while proposed post-training method restores motion conforming to parabolic Newtonian expectations.

Figure 8: Ramp slide down scenario: the physics-guided model preserves stable grounding and smooth deceleration, whereas other methods display floating, erratic surface contact, and non-smooth transitions.

(Figures 9–12)

Figures 9–12: Across free fall, ramp slide up, horizontal throw, and parabolic throw, only Newtonian-constrained post-training consistently yields physically coherent object trajectories, with correct acceleration and velocity patterns.

Implications and Future Directions

This work highlights the insufficiency of appearance-based or perceptual-feedback post-training for instilling physical structure in generative models. Explicit, verifiable constraints on measurable proxies, grounded in known physical laws, yield qualitatively improved temporal dynamics and robust generalization regimes. Importantly, the methodology generalizes—any physical law for which differentiable proxy variables can be estimated from video data is amenable to this framework. Broadening to richer forms of dynamics (non-constant forces, coupled systems, rotational effects, soft-body physics) is a logical next step. The creation of large-scale physics-benchmarks (e.g., NewtonBench-60K), and adoption of reward engineering for synthetic-to-real transfer, will accelerate development of world-simulators capable of predictive reasoning and physically reliable synthetic data generation.

Conclusion

The presented post-training protocol sets a new bar for physics-aware video generative modeling by enforcing Newtonian mechanics using verifiable reward signals. By bridging the gap between perceptual realism and physical consistency, this approach enables deployment of generative models in domains where dynamics matter—simulation, robotics, and scientific modeling. The use of measurable proxies and differentiable, rule-based rewards establishes a versatile, extensible foundation for the principled alignment of generative video synthesis with physical laws (2512.00425).