LingBot-World: Real-Time Simulation

• LingBot-World is an open-source world modeling framework that uses high-fidelity video diffusion and action-conditioning to simulate real-world dynamics.
• Its modular three-stage pipeline, featuring pre-training, action-conditioned middle-training with a Mixture-of-Experts, and causal distillation, enables real-time interactive performance at 16 fps.
• Benchmark evaluations show significant gains in dynamic degree and consistency, establishing LingBot-World as a versatile tool for robotics, content creation, and gaming.

LingBot-World is an open-source world modeling framework based on high-fidelity video diffusion and action-conditioning, supporting real-time interactive simulation across broad domains and extended temporal horizons. Originating from video generation research, LingBot-World is designed to provide robust, generalizable, and contextually consistent world simulation for applications spanning content creation, robot learning, and gaming. Its modular multi-stage pipeline, memory mechanisms, and causal dynamics architectures distinguish it as a foundation model for world simulation, interactive control, and vision-language research (Team et al., 28 Jan 2026).

1. System Architecture and Evolution

LingBot-World is engineered as a three-stage pipeline:

• Stage I (Pre-training): Initializes from a 14B-parameter Wan2.2 image-to-video diffusion generator, capturing strong spatiotemporal priors.
• Stage II (Middle-training): Converts the model into an action-conditioned world simulator by:
  • Expanding sequence context from seconds to minutes for long-term consistency.
  • Incorporating user actions via adaptive layer-normalization (AdaLN) into DiT transformer blocks.
  • Utilizing a Mixture-of-Experts (MoE) backbone (28B total parameters; inference cost ≈14B) to specialize computation for global scene structure (high-noise experts) versus local detail (low-noise experts).
• Stage III (Post-training): Distills the bidirectional model into a causal autoregressive architecture using block-causal attention and key–value caching, enabling real-time, stepwise rollouts at 16 fps.

The model’s autoregressive dynamics are formally characterized by maximizing the conditional log-likelihood over a horizon $L$:

$$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$

where $x_t$ are video frames and $a_t$ are control inputs (hybrid discrete-continuous). Action adapters modulate AdaLN in the transformer using Plücker-embedded camera controls and one-hot encoded keys (Team et al., 28 Jan 2026).

2. Fidelity, Dynamics, and Evaluation Metrics

LingBot-World is evaluated using standard and custom video generation metrics. For frame-level fidelity:

• PSNR: $10 \cdot \log_{10} (\mathrm{MAX}^2/\mathrm{MSE})$, where $\mathrm{MSE}$ is mean squared error.
• SSIM: Compares structural similarity via local statistics.
• FVD: Fréchet Video Distance, calculated on deep feature embeddings.

Dynamics and consistency are quantified by metrics including:

• Dynamic Degree: Custom differential score demonstrating temporal coherence.
• Frame Difference Consistency: $$DYN = 1 - \frac{1}{T}\sum_{t=1}^{T-1} \frac{\| \Delta x_t - \Delta \hat{x}_t\|_1}{\|\Delta x_t\|_1}$$
• Temporal Conditional Mutual Information: $TC = I(x_t;x_{t+k}|x_{t+1:t+k-1})$, optimized through bidirectional attention.

Empirical benchmarks show Dynamic Degree improvements of ∼16 points over prior models, with competitive motion smoothness. Ablative studies confirm that removal of action adapters collapses dynamic consistency (−30%), and omission of the adversarial head in distillation increases FVD by 22% and drops PSNR by 2 dB (Team et al., 28 Jan 2026).

3. Long-Horizon Memory and Causal Attention

Minute-scale horizon is achieved by:

• Contextual Curriculum: Training progresses from 5s to 60s video, improving context retention.
• Block-causal Attention: Video sequences are chunked; local (intra-chunk) attention is bidirectional, but inter-chunk connections are causally masked so information flows only forward.
• Key–Value Caching: States from previous chunks are cached for low-complexity access: updates follow

$$K_\ell = [K_{\ell-1}; W^K h_\ell],\quad V_\ell = [V_{\ell-1}; W^V h_\ell]$$

where $h_\ell$ are chunk hidden states.

These mechanisms enable LingBot-World to model and recall context over 10-minute rollouts with minimal drift, scaling favorably with sequence length (Team et al., 28 Jan 2026).

4. Real-Time Interactivity and Inference

Efficient causal distillation allows for <1 s latency when producing 16 frames (16 fps) on a single A100 GPU node. Complexity per newly generated frame is $$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$0 (for chunk size $$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$1 and transformer width $$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$2) due to key–value caching, in contrast to $$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$3 for naive bidirectional models. This supports real-time agent interaction and user control within open simulation environments (Team et al., 28 Jan 2026).

Integration with an interactive Python API enables stepwise control: agents can accept user actions (e.g., discrete navigation or continuous camera) and render environment changes in real time:

$$\max_\theta\ \mathbb{E}\left[\log p_\theta(x_{t:t+L} \mid x_{<t}, a_{t:t+L})\right]$$4

5. Domain Generalization and Environment Breadth

LingBot-World is pre-trained on a corpus encompassing:

• Realistic first- and third-person videos.
• Scientific visualizations (molecular, astrophysical).
• Artistic and synthetic content (cartoon, pixel art, steampunk).
• Game-engine trajectories (Unreal Engine, both scripted and human demonstration).

Hierarchical captioning, multi-expert specialization, and progressive curriculum drive generalization, without recourse to explicit domain-adaptation losses. Generalization to novel prompts is supported by the underlying text-conditional video backbone (Team et al., 28 Jan 2026).

6. Applications, Benchmarks, and Open Release

Practical impact spans content creation, simulation for robotics learning, and gaming. Comparative results (Table 2 in (Team et al., 28 Jan 2026)) on VBench show LingBot-World is the only open-source, general-domain model simultaneously achieving high dynamic degree (0.8857 at 720p, +16% absolute over competitors), aesthetic quality, and overall consistency at real-time rates. Removal of core architectural elements in ablation leads to degraded dynamics and perceptual metrics.

All code, model weights, and checkpoints are released under the Apache 2.0 license. Public resources include website, GitHub, and HuggingFace repositories for reproducible research and downstream application.

Model	Imaging Q.	Dynamic Degree	Consistency
Yume-1.5	0.5838	0.7612	0.1994
HY-World 1.5	0.6512	0.7217	0.2016
Ours	0.6683	0.8857	0.2178

7. Extensions and Relation to Causal World Models

LingBot-World supplies the foundational video-action modeling upon which systems such as LingBot-VA build. LingBot-VA extends this framework with a shared latent space, Mixture-of-Transformers dual-stream architecture, closed-loop rollout, and asynchronous inference, enabling simultaneous visual prediction and policy action for robot control (Li et al., 29 Jan 2026). This highlights the role of robust, causally grounded video world models as core enablers of sample-efficient, generalizable vision-language-action interaction in simulation and real-world tasks.

LingBot-World thus serves as a core infrastructure for embodied world modeling, supporting research in vision-language navigation, emergent communication via world models, and interactive simulation agents (Team et al., 28 Jan 2026, Li et al., 29 Jan 2026, Cowen-Rivers et al., 2020, Yan et al., 2019).