Infinite Worlds with Versatile Interactions

Abstract: We present LingBot-World 2.0 (also known as LingBot-World-Infinity), an advanced iteration of LingBot-World featuring four distinct upgrades. (1) Our model achieves an unbounded interaction horizon while maintaining consistent output quality, benefiting from a carefully crafted causal pretraining paradigm. (2) Through distilling a real-time variant from the base model, our system guarantees rapid response time, sufficient to drive 720p video streams at 60 fps. (3) Compared to the previous version, this update introduces highly diverse interactive elements, comprising a broader spectrum of actions (e.g., attacking, archery, spell-casting, and shooting) alongside a richer variety of text-driven events. (4) We pioneer the integration of an agentic harness within the domain of world modeling, wherein a pilot agent is tasked with planning and executing character behaviors, while a director agent is responsible for synthesizing novel environmental elements as the scene progresses. Additionally, to facilitate a shared experience, we develop an interface that permits multiple players to simultaneously immerse themselves in this vivid world simulator. We pair our primary 14B model with a lightweight 1.3B counterpart, which supports effortless deployment on a single GPU.

• The paper introduces a rigorously causal video synthesis framework that minimizes error accumulation to achieve hour-long, high-resolution stable world generation.
• It leverages a unified data pipeline with temporal segmentation and hierarchical annotations to enable responsive, action-conditioned video simulations.
• The system integrates a dual-agent harness combining a director’s semantic reasoning with a pilot’s frame-by-frame synthesis to support interactive control.

Infinite Worlds with Versatile Interactions: Technical Analysis and Implications

Overview and Motivation

"Infinite Worlds with Versatile Interactions" (2607.07534) introduces a robust interactive video world modeling framework with notable advances in long-horizon stability, real-time performance, broadened interactivity, and agentic orchestration. The central objective is to overcome persistent drift and responsiveness limitations inherent in prior causal (autoregressive) video generative models, thereby enabling the synthesis of infinite, explorable environments that support rich semantic and physical interactions at 720p/60fps, without perceptible degradation over hour-long rollouts.

Figure 1: The system generates infinite worlds in real time, featuring versatile interactions.

The work augments the state of the art by (i) deploying a rigorously causal pretraining protocol to mitigate error accumulation across temporal rollouts, (ii) distilling a high-throughput, low-latency model for interactive deployment, (iii) expanding the interaction space to transcend navigation, introducing action primitives such as combat and spell-casting, and (iv) pioneering a “pilot-director” agentic harness where a VLM (Director) and a video model (Pilot) enable semantically-grounded, user-driven world trajectories.

Data Engine and Annotation Pipeline

To train a model capable of action-conditioned, temporally grounded simulation, the authors construct a unified data pipeline ingesting multi-source egocentric and synthetic videos, complemented by extensive web-mined content. The pipeline employs sequential (i) quality and content profiling (technical + VLM filters), (ii) temporal segmentation, and (iii) hierarchical annotation. Critically, each video is annotated both with global captions and chunk-wise, temporally localized semantic descriptions, decoupling scene, event, interaction, and environment dynamics.

Figure 2: Heterogeneous raw videos are temporally segmented, filtered, and routed to category-specific annotation pipelines, yielding optimized chunk-wise captions.

This annotation strategy is central for bridging the train–inference gap: whereas typical models only see global scene context, this approach exposes the model to temporally-indexed conditioning signals, enabling robust response to time-varying user interactions at inference.

Generative Model Architecture and Training

The simulation problem is formalized as a strictly causal process: at each step, the world state is generated from past visual observations and current user actions (comprising camera pose and chunk-wise prompt inputs). The model predicts the next frame $x_t$ given context $(x_{<t}, a_{\leq t})$:

$$p_\theta(x_{1:T} \mid a_{1:T}) = \prod_t p_\theta(x_t \mid x_{<t}, a_{\leq t}).$$

Figure 3: Pipeline overview—an initial image and background, then autoregressive world state generation (causal video model) conditioned on historical context and user inputs.

MoBA Attention Mechanism

A key architectural innovation is the Mixture of Bidirectional and Autoregressive (MoBA) attention mask. Traditional teacher-forcing facilitates autoregressive training but causes over-reliance on context, leading to degradation at long horizons. The MoBA mask integrates a fully bidirectional block into teacher-forcing, regularizing context usage and promoting visual persistence while preserving causality.

Figure 4: The DiT block with MoBA Attention Mask integrates camera poses and prompts, balancing autoregressive and bidirectional contexts for visual fidelity.

The action space embeds camera control (via Plücker embeddings and AdaLN) and temporally resolved prompts into the diffusion backbone, supporting high-granularity user influence and multiscale semantic control.

Post-Training Distillation

To enable real-time interactivity, the base model is distilled using a hybrid of Consistency Distillation (for few-step generation) and Distribution Matching Distillation (DMD; for aligning the student’s long-horizon rollout distribution to teacher/data, reducing compounded drift). Notably, DMD is performed over model-generated rollouts, ensuring robustness to distribution shift during autoregressive sampling.

Agentic Harness and System Deployment

To transition from passive video synthesis to interactive simulation, the authors implement a co-simulation harness:

• Director (VLM): Performs causal reasoning, inspects scene context, and generates semantic “event proposals” (actions, interventions, or environmental shifts).
• Pilot (Video Generator): Grounds these high-level events into physically coherent, frame-by-frame video rollouts.

  Figure 5: Overview of the Agentic Interaction Harness, enabling semantic, object-centric, and high-level interventions via Director-Pilot co-simulation.

In tracking mode, a SAM-based loop localizes and tracks objects, enabling precise, persistent manipulations and context-aware actions.

Figure 6: The tracking-mode interface leverages VLM for object comprehension and action proposal; the pilot renders causally coherent dynamics in response.

The system supports highly versatile interactions, ranging from direct semantic actions (navigation, object manipulation) to prompt-driven interventions (weather changes, entity injection), with seamless switching of context and protagonist across world horizons.

Figure 7: Controllable world exploration—supports flexible prompt switching, evolving semantics, and protagonist navigation.

Figure 8: Continued demonstration of prompt/control switching and multi-entity navigation.

System Optimization and UI

Interactive real-time performance is achieved via inference stack optimizations (compiler-level, streaming, multi-GPU pipeline), a spatio-temporal refiner (for spatial upsampling and temporal interpolation), and dynamic KV-cache management (to retain maximal relevant historical context per chunk, enhancing both quality and throughput). The user interface integrates low-level movement/view controls with VLM-generated semantic event triggers, enabling direct and context-dependent interaction scripting.

Figure 9: Interactive system interface with real-time viewport, low-level WASD/IJKL controls, and hotkey-mapped VLM event proposals.

Empirical Results and Comparative Analysis

Quantitative and qualitative benchmarks demonstrate that the distilled 14B/1.3B models are the only open-source systems sustaining hour-level (practically infinite) generation at high resolution and real-time rates, without visible scene drift or decay (in sharp contrast to minute-scale prior models).

Figure 10: Hour-long generation—frames from a single uninterrupted 60-minute session in 20 distinct scenarios, showcasing long-horizon visual and structural coherence.

Comparative evaluation shows the system matches or exceeds both open and closed-source baselines in temporal stability, visual fidelity, responsiveness, and controllable interaction breadth.

Figure 11: Qualitative comparison—model maintains stable visual and physical consistency over long-horizon generation.

Additionally, the causal pretraining backbone outperforms leading alternatives in preventing accumulation-induced drift under autoregressive sampling.

Figure 12: Backbone causal pretraining maintains stability relative to prior models under extended rollouts.

Implications, Limitations, and Future Directions

Practical implications: This system provides a fully open, high-fidelity, drift-resistant world model that supports semantically and physically meaningful interactions in real time—a substantial advance for research in generative environments, simulation learning, and interactive storytelling. Potential applications include embodied AI, simulation-based decision-making, procedural content generation, multi-agent interaction studies, and reinforcement learning environments.

Theoretical implications: The methodology establishes that with carefully designed causality constraints, context management, and hybrid attention, error accumulation in vision-only autoregressive world models can be fundamentally reduced. Furthermore, coupling a semantic reasoning VLM and a diffusion pilot demonstrates the viability of compositional, agentic generative simulation.

Limitations:

• The model’s long-term persistence is only apparent—once a region leaves the active context, it is regenerated rather than faithfully recalled, limiting identity and relational consistency.
• Subtle identity/style drift can occur over extended play.
• The current model lacks explicit geometric/physical priors; thus, catastrophic physical impossibilities (e.g., object interpenetration) can occur.
• Despite efficient distillation, achieving real-time high-res generation on commodity hardware remains a challenge.

Directions for Future Research: Integrating compact, scalable memory architectures for persistent world state, enforcing entity/identity and style invariance, and incorporating explicit physics models or geometric constraints are likely areas for future breakthroughs. Additionally, further reducing the computational footprint for broader accessibility is a critical engineering problem.

Conclusion

This work presents a rigorously engineered framework for infinite, interactive video world generation with strong resistance to temporal drift, broad action controllability, and real-time latency. The dual-agent harness, architectural innovations in causality and context, and the open-source release position this model as a significant platform for downstream research across interactive AI, visual reasoning, and synthetic environment generation (2607.07534).