Long-horizon consistency in interactive generative world models

Establish mechanisms for sustaining long-horizon consistency in interactive generative systems, including action-conditioned video and scene-based world simulators, so that object identity, context, and coherence are preserved over extended real-time interaction.

Background

Stage III advances world models from static generation to real-time, action-conditioned simulation. Despite this progress, models commonly lose context and hallucinate over extended play, revealing brittle long-term behavior.

The authors note a tension between explicit 3D scene generators (e.g., NeRFs, Gaussian Splatting) that offer spatial stability but require explicit modeling, and implicit frame-by-frame generators that are flexible yet prone to drift and forgetting. They explicitly state that sustaining long-horizon consistency remains unsolved.

References

Despite the leap to real-time interaction, sustaining long-horizon consistency remains unsolved.

— From Masks to Worlds: A Hitchhiker's Guide to World Models  (2510.20668 - Bai et al., 23 Oct 2025) in Section 5.3 (Challenges)

Few-step generation, distillation, and streaming are used to reduce latency, while action semantics and persistent state remain open long-horizon problems \citep{he2025matrixgame2,forgewm2026}.

— AI for Games in the Foundation Model Era  (2609.16679 - Luo et al., 15 Sep 2026) in Section 4.4, “Action Interfaces, Real-Time Generation, and Long-Horizon Consistency”

Several directions remain open. First, the current geometric world state primarily preserves coarse scene structure, while fine-grained consistency of object identity, appearance, and local details remains limited. Richer object-level or semantic world representations may provide stronger long-term identity consistency.

— Alaya-EVOKE: From Linear-Scaling Supervision to Endless World  (2608.13546 - Yin et al., 13 Aug 2026) in Conclusion

Several directions remain open. First, the current geometric world state primarily preserves coarse scene structure, while fine-grained consistency of object identity, appearance, and local details remains limited. Richer object-level or semantic world representations may provide stronger long-term identity consistency. Second, a persistent world should model not only static geometry but also dynamic state, including object motion, state transitions, and their long-term evolution. Developing explicit representations that can continuously update such dynamic world state is an important next step.

— Alaya-EVOKE: From Linear-Scaling Supervision to Endless World  (2608.13546 - Yin et al., 13 Aug 2026) in Conclusion

However, this raises a fundamental question: can generative world models maintain persistent beliefs about unobserved objects and agents, reason about physical constraints beyond the training distribution, and provide uncertainty-aware predictions that remain consistent over long-horizon rollouts? Addressing these challenges remains an open problem and is essential if world models are to provide a foundation for scalable robot learning, motivating the open problems discussed in Section \ref{sec:open_problems}.

— Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models  (2609.03927 - Mehta et al., 3 Sep 2026) in Discussion and Challenges following Section 6.4, World Models as Simulation Engines

Finally, while OctWorld demonstrates robust capabilities in the long-range generation of static scenes, it does not yet support the generation and memory retention of dynamic objects. How to effectively model 4D scenes and maintain dynamic world memory remains a critical open challenge.

— OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping  (2609.03919 - Lv et al., 3 Sep 2026) in Appendix, Section “Failure Case, Limitations and Future Work,” subsection “Limitations and Future Work”

Moreover, ensuring stability over long horizons remains difficult, as small errors in latent prediction often accumulate, leading to compounding deviations that degrade policy performance.

— A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform  (2610.00926 - Xu et al., 1 Oct 2026) in Section 4.5, “World Models”