Constructing a world model that fully clones deterministic 3D environments
Construct a world model that, trained purely from first-person observation–action trajectories, can fully clone and overfit a fixed deterministic 3D environment by reproducing its unique ground-truth sequence of observations for any given action sequence over long horizons.
References
In this work, we take a step toward building a truly accurate world model by addressing a fundamental yet open problem: constructing a model that can fully clone and overfit to a deterministic 3D world.
The proxy itself also matters: outputs sometimes follow the input geometry too closely and inherit the coarse shapes of low-poly proxies, while box proxies leave open which described appearance belongs to which entity, so designing a good proxy remains an open problem.