Learning and reasoning over long-horizon vision–language sequences

Establish effective methods to learn from and reason over long-horizon interleaved vision–language sequences that capture extended temporal dynamics and semantic coherence.

Background

The paper positions long-horizon multimodal learning—particularly from interleaved vision–language sequences such as long videos with aligned transcripts—as a key capability for world models. While short-clip generation has advanced, sustaining temporal consistency and coherent reasoning over extended sequences remains challenging. Emu3.5 is proposed as a native multimodal next-token predictor trained on large-scale interleaved data to move toward this goal, but the authors explicitly acknowledge this remains open.

Addressing this problem is central to building world models that can generalize across time, maintain consistency of entities and narratives, and integrate visual and linguistic context over many steps. It underpins downstream capabilities such as visual narrative, visual guidance, world exploration, and embodied manipulation.

References

Recent advances in short-clip video generation have demonstrated the ability to capture short-term dynamics, but learning from and reasoning over long-horizon vision-language sequences remains a central open challenge.

— Emu3.5: Native Multimodal Models are World Learners  (2510.26583 - Cui et al., 30 Oct 2025) in Section 1 (Introduction)

The open question is no longer whether models can be made to reason over time, but whether certificate-long horizons can be reached under the compute budget of a wearable rather than a server.

— Vision-Language Models for Egocentric Video: From Hand-Object Interaction to Embodied AI  (2608.18671 - Zamani et al., 19 Aug 2026) in Section 11.1, “Better Temporal Reasoning” (Sec. future-temporal)

Whether the registers can also carry the much longer context of tasks that depend on events far in the past remains to be tested.

— REKI: Registering Knowledge in VLAs  (2610.22178 - Marrie et al., 15 Oct 2026) in Section 7, Discussion, subsection “Memory”