PhysMaster: Physics-Driven Video Generation

• PhysMaster is a reinforcement learning-driven framework for video generation that integrates explicit physical knowledge to yield dynamic, physics-consistent video sequences.
• It combines a PhysEncoder that extracts scene-level physical information with a transformer-based video generator to maintain temporal and semantic consistency.
• Experimental results demonstrate improved IoU, L2 trajectory accuracy, and realistic object interactions compared to conventional video synthesis methods.

PhysMaster is a reinforcement learning-driven framework for video generation that explicitly targets the induction of physics-consistent behavior in generative models. Unlike conventional video synthesis approaches that tend to prioritize visual plausibility and coherence, PhysMaster is designed to integrate explicit physical knowledge by conditioning the generation process on representations that encode the underlying dynamical laws and constraints—addressing the longstanding challenge of producing physically plausible “world models” in open-ended generation tasks (Ji et al., 15 Oct 2025).

1. Motivation and Conceptual Overview

PhysMaster addresses the observed gap in contemporary video generation architectures: existing models, despite being visually convincing, routinely violate conservation laws, exhibit implausible object interactions, and fail to extrapolate meaningful physics in new scenes. The key insight of PhysMaster is to introduce a path from static images to physically plausible video dynamics by learning physical representations from the initial frame and incorporating these representations as conditioning variables into the video generative process.

The central problem formulation is an image-to-video task: given a single image (encoding a scenario with multiple objects, their positions, and implicit cues about their possible interactions), the model must generate a temporally consistent and physically lawful video sequence. The physical knowledge extracted from the input is used—via the so-called PhysEncoder—to guide the entire video rollout.

2. Model Architecture and Information Flow

PhysMaster is constructed around two main architectural innovations:

• PhysEncoder: A module responsible for extracting and encoding scene-level physical information from the initial image. The encoder backbone is based on the DINOv2 vision transformer, producing semantic-rich feature maps, which are processed by a domain-specific trainable “physical head” to obtain a compact embedding $c_{phys}$ representing relevant physical state and priors.
• Video Generator (DiT backbone): A transformer-based diffusion model for video synthesis. Latent codes representing frame content are generated sequentially, each step being conditioned on a tuple $(c_{text}, c_{image}, c_{phys})$, where $c_{text}$ is the embedding of the textual prompt, $c_{image}$ encodes the initial frame, and $c_{phys}$ encapsulates the physical priors extracted by PhysEncoder.

The fusion of these representations enables the DiT model to align the generated frame transitions (i.e., video dynamics) with both semantic intent and physics-derived constraints.

Training proceeds via supervised and reinforcement learning stages:

• Supervised Fine-Tuning (SFT): Initially, both the DiT and PhysEncoder modules are trained jointly using a flow matching loss that encourages the predicted velocity field in latent space, $v_\theta(z_t, t, ...)$, to approximate the observed change between start and target frames:

$$\mathcal{L}_{lcm} = \mathbb{E}_{t, z_0, \epsilon} \left\|v_\theta(z_t, t, c_{text}, c_{image}, c_{phys}) - (z_1 - z_0)\right\|_2^2$$

• Reinforcement Learning with Direct Preference Optimization (DPO): To overcome the inadequacy of pixel-wise or trajectory-only loss in enforcing physics, PhysMaster adopts a reinforcement learning with human feedback (RLHF) pipeline:
  • Stage II – DPO for DiT: The DiT model is further aligned using pairwise feedback (preference between two generated videos under the same prompt and image but distinct seeds). Preferences reflect physical plausibility, and reward signals guide the optimization of DiT, subject to KL constraint with a reference model.
  • Stage III – DPO for PhysEncoder: The feedback from improved DiT generations is leveraged to adaptively refine the physical head of PhysEncoder, such that the physical embedding $c_{phys}$ increasingly encodes those scene attributes most predictive of physically correct evolution.

3. Physical Representation Learning Mechanism

PhysEncoder’s objective is to map the first video frame to a representation encompassing both explicit physical states (positions, geometry, and apparent constraints) and implicit environmental factors (e.g., gravity direction, friction, support conditions). Importantly, supervision is indirect: PhysEncoder is updated to maximize preference-aligned outcomes mediated by the DiT model, facilitated by Flow-DPO objectives of the form:

$$-\mathbb{E}\left[\log \sigma\left( -\frac{\beta}{2}\left(\|v^w - v_\theta\|^2 - \|v^l - v_\theta\|^2 - [\text{reference terms}]\right)\right)\right]$$

where $v^w$ and $v^l$ are the velocity fields corresponding to the preferred and less preferred generations, respectively.

Principal component analysis (PCA) of the physical embeddings demonstrates clustering by physical state (free-fall vs. grounded), confirming that the learned representation robustly encodes scene-level forces and constraints relevant to downstream dynamics.

4. Reinforcement Learning Framework: Direct Preference Optimization

The introduction of DPO in PhysMaster is critical for converting weak, preference-based physics supervision into an effective signal for both the generator and the physical encoder. The generic DPO objective is:

$$\max_{\pi_\theta} \mathbb{E}_{p, i, x}[ r_\phi(x, p, i) ] - \beta D_{KL}\left(\pi_\theta(x\mid p, i) \| \pi_{ref}(x\mid p, i) \right)$$

where $r_\phi$ is the reward corresponding to physics consistency and plausibility, and $\pi_{ref}$ acts as a regularization anchor. The preference-based construction ensures optimization cannot exploit spurious visual cues, focusing instead on genuine physical congruence as perceived by human annotators or algorithmic preference models.

In the full PhysMaster pipeline, both the DiT video generator and (in a final stage) PhysEncoder are tuned via these DPO objectives, cementing the physical inductive bias throughout the model hierarchy.

5. Experimental Evaluation and Results

PhysMaster was benchmarked on restricted proxy tasks and general, open-world physical scenarios:

• On controlled “free-fall” tasks (Kubric dataset), PhysMaster achieves higher Intersection-over-Union (IoU, measuring shape consistency) and strong semantic adherence compared to methods such as PhysGen and PISA, while maintaining competitive trajectory L2 accuracy and Chamfer Distance (CD).
• In broader evaluations across diverse physical phenomena (Dynamics, Thermodynamics, Optics), metrics of semantic adherence (SA) and physical commonsense (PC) confirm that full reinforcement learning—especially DPO applied to PhysEncoder—substantially improves both physical plausibility and generation efficiency over naive SFT or iterative RL-only baselines (e.g., PhyT2V).
• Ablation studies reveal that the RL alignment for the physical encoder (stage III) is essential: omitting this results in reduced physical realism, degraded object rigidity, and implausible motion.

6. Generalizability, Modularity, and Implications

A defining characteristic of PhysMaster is the modularity of its physical guidance. Since the physical embedding is disentangled from specific kinematic or environmental details, the same PhysEncoder can, in principle, be plugged into other diffusion- or transformer-style video generation backbones (or even employed in simulation-based inference tasks requiring physics priors). This suggests applicability for robotics policy learning, AR/VR physics simulation, and as a physics-informed module in broader multimodal systems.

PhysMaster’s paradigm—learning physical representations via indirect, reinforcement-driven preference signals—also provides a blueprint for similar approaches in text, image, and language domains requiring domain-specific inductive biases beyond raw data statistics.

7. Key Mathematical Formulations

Component	Mathematical/Algorithmic Formulation	Role
Flow Matching Loss	$$\mathcal{L}_{lcm} = \mathbb{E}\| v_\theta(z_t, ...) - (z_1-z_0)\|^2_2$$	Aligns DiT-predicted velocities with ground-truth motion
RL/DPO Objective	$$\max_{\pi_\theta} \mathbb{E}[ r_\phi ] - \beta D_{KL}(\pi_\theta || \pi_{ref})$$	Preference-guided RL for physical validity
Flow-DPO Preference Obj	$-\mathbb{E}[ \log \sigma( -\frac{\beta}{2}(...)) ]$	Enforces alignment to preferred trajectories/velocities

Conclusion

PhysMaster establishes a new standard in physics-consistent video generation by unifying supervised dynamics learning with reinforcement learning from human preference feedback. It does so via an explicit physical representation pathway, trained through DPO to encode scene-level physical knowledge and propagate it through the generative process. The result is a substantial advance in the generation of videos that not only exhibit visual fidelity but also exhibit lawful, generalizable physical behavior—a foundational step toward scalable, physics-aware world models for AI (Ji et al., 15 Oct 2025).