Interaction dynamics in latent-predictive world models

Determine whether latent-predictive world models learn to capture interaction dynamics more readily than generative world models.

Background

The paper evaluates generative world models that produce future pixels and measures interaction fidelity through segmentation-based metrics for agents and manipulated objects. Its experiments show that scaling ego-centric human video improves agent fidelity substantially more than the fidelity of the objects affected by the agent, and that the proposed dynamic-region supervision only partially raises the object-fidelity ceiling.

The authors note that latent-predictive models may offer a different route to learning interaction dynamics, but such models do not render pixels and therefore cannot be assessed using the paper’s fidelity metrics. Whether latent-predictive formulations learn interaction dynamics more readily consequently remains unresolved.