Video = World + Event Stream

Abstract: We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a video is a world plus an event stream. The world is the persistent context in which a video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and other relatively stable conditions. The event stream is everything that changes over time within that world, including scene or environmental changes, subject behavior, speech, and other sounds. This yields a general-purpose pretraining task over large amounts of real video: given a world and incoming input, predict how the world moves, changes, and responds in real time. The resulting competence can be specialized to a broad family of real-time downstream tasks. We instantiate it on real-time full-duplex audio-visual interaction, where the event stream is the agent's speech together with free-form behavior. Functionally, the model's multimodal understanding process is vision-language-action-like: it maps multimodal user input to language-form speech and behavior actions. Wan-Streamer v0.3 preserves the v0.2 operating point: 640x368 video at 25 FPS, a 160 ms streaming unit, approximately 200 ms model-side response latency, and approximately 550 ms total interaction latency under a 350 ms bidirectional network budget.

• The paper introduces a novel factorization that decomposes video into a persistent world context and a time-indexed event stream for real-time interaction.
• It details an integrated Transformer architecture that synchronizes language, audio, and video streams to maintain strict temporal consistency.
• The study demonstrates that the approach preserves low latency (approximately 550 ms total delay) while expanding expressive multimodal capabilities.

"Video = World + Event Stream": Authoritative Analysis

Introduction

"Video = World + Event Stream" (2607.15038) introduces Wan-Streamer v0.3, which conceptualizes video as the combination of a persistent world context and a time-indexed stream of events. This decomposition is instantiated as a unified, end-to-end streaming model for real-time, full-duplex audiovisual interaction. The paper frames its approach as a substantial abstraction improvement over previous system designs (Wan-Streamer v0.1/v0.2), supporting expanded capabilities while strictly preserving response latency and deployment topology. The following essay provides an expert synthesis of the core technical advances, experimental findings, and broader implications, with an emphasis on the transition to a world + event stream paradigm.

World + Event Stream Factorization

The essential contribution of Wan-Streamer v0.3 is the explicit separation of a video into (1) a persistent world $W$, comprising static contextual information (environment, scene, character states, visual/acoustic background, persona), and (2) an event stream $\{e_k\}$, denoting all causally evolving entities (motion, speech, actions, environmental events). The world context is represented as a structured, clip/task-dependent record, while events are temporally indexed records with context alignment and optional character association. This decomposition supports generalization and modularity in multimodal pretraining.

Figure 1: Example pretraining data: time-aligned events (actions, speech, and motions) paired with world context summaries, permitting robust multimodal decoding and alignment.

Formally, the probability over a sequence of event units conditioned on the world and the observed input is autoregressive:

$$p_\theta(e_{1:K} \mid W, x_{1:K}) = \prod_{k=1}^{K} p_\theta(e_k \mid W, x_{\leq k}, e_{<k}).$$

During pretraining, large-scale video corpora provide both $W$ and the ground-truth event stream, allowing the model to learn context-conditional event progression dynamics at scale. At inference, event streams are produced in response to real-time user input and the prefilled world context.

Multimodal Streaming Interaction

The architecture maintains a joint, interleaved causal timeline for all modalities (language, audio, video, user/agent events), unified in a single Transformer model. This allows perception, speech, visible listening, and behavioral responses to be synthesized with strict temporal consistency, leveraging block-causal attention and efficient context tokenization.

Figure 2: The world context is prefilled; audio, video, and language I/O are streamed on a synchronized causal timeline, with block-causal attention and token-based behavior/speech actions directly conditioning audio-video synthesis.

Behavior actions are specified as natural-language "directives" interleaved within the text output stream, e.g., (nods) Yes, I heard you.. This approach enables open-vocabulary, context-grounded behavior generation, as opposed to fixed, small action vocabularies. Both events and speech are tokenized for next-token prediction, conditioning real-time audio-video latent generation. Critically, behavior timing and ordering are now model-composed, not post-processed or controlled externally.

Latency and Fidelity Benchmarks

Despite significant architectural and objective changes, Wan-Streamer v0.3 preserves the runtime envelope of Wan-Streamer v0.2: 640×368 video at 25 FPS, with 160 ms streaming slice, ~200 ms model-side latency, and ~550 ms total end-to-end streaming delay (including a 350 ms bidirectional network budget). These figures are invariant under the new world + event stream formulation, as the underlying model contract and inference path remain unaltered.

Expressive Range

Qualitatively, v0.3's agents are capable of producing coherent, context-driven open-vocabulary actions, maintaining temporal congruence with speech and environmental cues. Idleness, attentive listening, and dynamically described motions (e.g., "smiles briefly," "reaches for object") are rendered in a manner tightly synchronized with speech prosody and scene context. Compared with prior versions limited to visible listening and implicit gestures, v0.3 permits a substantially richer and more controllable agent action space.

Theoretical Implications and Comparison

General-Purpose Pretraining

The explicit world + event factorization provides a scalable and generalizable framework for multimodal sequence modeling. By decoupling persistent context from event progression, the pretraining task naturally generalizes beyond conversational avatars to world-model-based control, navigation, and embodied manipulation—domains explored in parallel works such as "FlowAct-R1" (Wang et al., 15 Jan 2026), "Matrix-Game 3.0" (Wang et al., 10 Apr 2026), and "WorldPlay" (Sun et al., 16 Dec 2025). Conditioning on persistent world context allows for the specialization of event prediction logic over diverse real-time tasks without retraining core sequence dynamics.

Comparison with Related Work

Recent avatars and streaming multimodal agents—e.g., LiveTalk (Chern et al., 29 Dec 2025), StreamAvatar (Sun et al., 26 Dec 2025), Midas (Chen et al., 26 Aug 2025), X-Streamer (Xie et al., 25 Sep 2025)—typically adopt a single causal sequence or explicitly factor speech and action, without an equivalent contextual world decomposition. Approaches like DuplexSLA (Zhang et al., 20 May 2026) and SALM-Duplex (Hu et al., 21 May 2025) offer synchronized language-speech-action modeling, but remain domain-specialized (dialogue, synthetic benchmarks), lacking the abstract world-event modularity that enables transfer across real and synthetic interaction settings.

The model's open-vocabulary language-for-behavior interface is functionally analogous to vision-language-action (VLA) planning in embodied agents, but with outputs realized at high-fidelity real-time rates and grounded by synchronized audiovisual generation.

Practical Implications and Future Directions

This modeling framework facilitates rapid scaling and adaptation for new downstream interfaces by modifying only the world context schema and the input stream, preserving efficient training and serving. Practically, the architecture can specialize to navigation (alter world fields, alter event schema), physical manipulation, or highly expressive multimodal human-avatar interaction with minimal modifications.

This decomposition could catalyze new forms of multimodal world models, reinforcement learning agents, and embodied simulators. The persistent context as a first-class conditioning object also aligns with advances in context-driven long-horizon video modeling (Gu et al., 25 Mar 2025), consistent interactive generation [ICCV 2025], and real-time world modeling (Sun et al., 16 Dec 2025, Wang et al., 10 Apr 2026).

Future work may extend the evaluation to embodied control, systematic domain transfer, or interactive 3D world synthesis, leveraging the same abstraction and model contract. Integration with retrieval-augmented context construction, hierarchical policy learning, or physical simulation may be explored.

Conclusion

Wan-Streamer v0.3 recasts streaming audiovisual interaction as sequence modeling over a (world, event stream) pair, decoupling persistent context from dynamic event generation. This abstraction supports broad, scalable pretraining and specialization for real-time interaction tasks without compromising responsiveness or fidelity. The world + event framework provides a theoretically sound and practically extensible foundation for multimodal video generation, embodied simulation, and real-time agent research.