Vidu S2: Real-Time Interactive Video That Edits Itself
Vidu S2 breaks the offline barrier in diffusion video generation by introducing a unified system for real-time interactive avatar synthesis and streaming video editing. Through causal diffusion adaptation, Self-Replay Forcing, and VLM-mediated state management, the system generates 720p audio-driven characters and applies style transfer, virtual try-on, subject replacement, and background replacement to live streams while maintaining temporal stability and identity consistency across extended sequences.Script
Most video generation models produce a single clip and stop. Vidu S2 keeps going, letting you change references, edit style, swap clothing, and replace backgrounds while the video runs, all without breaking temporal coherence or losing identity.
The core challenge is the train-test mismatch. Training sees clean sequences, but streaming inference must condition on its own noisy history. Errors compound, causing identity drift, motion discontinuity, and eventual collapse.
Self-Replay Forcing addresses this by generating a long rollout, detaching it, then replaying the noised segments with gradients flowing across time. Combined with preference optimization and a resolution refiner, the system reaches 720p at 25 to 42 frames per second.
Vidu S2-Avatar supports dynamic references during generation. A VLM agent interprets your instruction, observes the resulting frames, and maintains persistent state, so when the character picks up a cup or puts on a hat, later prompts preserve that fact unless you change it.
Vidu S2-Editing performs frame-aligned editing, attending only to the source frame at the same temporal position. This preserves motion and timing while transforming style, trying on garments, swapping subjects, or replacing backgrounds across more than 50 styles.
On StreamAV-Bench, Vidu S2-Avatar leads every metric, and human raters confirm sustained quality across 90-second streams. The system demonstrates that diffusion video can move from one-shot synthesis to continuous interaction. Learn more and create your own videos at EmergentMind.com.
