- The paper introduces ViMax, a multi-agent video-generation framework that combines hierarchical story decomposition, retrieval-augmented planning, dependency-aware scheduling, transition videos, and VLM-based candidate selection.
- The system improves overall global consistency on ViMax-Bench from 0.527 with the strongest baseline to 0.557, while achieving a 77.62% win rate over StoryDiffusion in human evaluations.
- The paper shows that cross-shot visual dependencies are crucial: removing them reduces global consistency by 8.7%, while event-aware planning substantially improves narrative faithfulness from 2.34 to 3.52 in novel-to-video adaptation.
Overview
ViMax is a multi-agent framework for long-form video generation that addresses two failure modes of current text-to-video systems: the absence of narrative structure beyond isolated short clips, and visual drift in character identity, environment, and spatial layout across shots. The framework decomposes production into specialized agents—screenwriting, shot planning, character styling, video generation, and VLM-based quality control—and introduces two technical mechanisms: hierarchical story decomposition with retrieval-augmented generation (RAG) for global narrative context, and graph-based visual dependency tracking combined with transition video generation for cross-shot consistency. The authors also release ViMax-Bench, a 35-story benchmark targeting controlled cross-shot consistency challenges (2606.07649).
Motivation and problem setting
Existing approaches fall into three categories, each with structural shortcomings. Streaming methods such as StreamingT2V and FreeLong extend single-clip duration but cannot handle multi-scene narratives. Consistency mechanisms such as StoryDiffusion, ConsiStory, and STAGE preserve subject identity through shared attention or keyframe anchoring but lack narrative planning. Multi-agent frameworks including FilmAgent, Kubrick, and VideoGen-of-Thought decompose production into roles but do not coordinate narrative, visual, and temporal constraints jointly. The paper frames the core difficulty as two coupled challenges: multi-shot visual consistency (object-level identity plus spatial coherence across camera angles) and long-form narrative planning that exceeds LLM reasoning capacity when treated monolithically.
Methodology
Agentic workflow
The pipeline begins with user input I (a few sentences, a novel, or a screenplay draft), which a categorization mechanism routes to an appropriate prompt template for LLM-based screenwriting. A storyboard agent then converts the screenplay into shot specifications S1,…,Sn covering camera angles, movements, lighting, positioning, and duration. Generation follows a two-step text-image-video process: key images (character profiles, scene backgrounds, shot keyframes) are produced first, then text-and-image-to-video models render each shot. At every stage, VLM-based quality control samples k candidates in parallel and selects the highest-scoring one against the shot specification along visual fidelity, narrative consistency, and specification adherence.
Hierarchical narrative planning with RAG
Long-form planning is handled by recursive decomposition: screenplay → story events → scenes → shots, so each LLM call operates on an appropriately scoped task. The paper acknowledges that decomposition discards rich contextual detail (character motivations, thematic links), which it recovers via RAG: the source material is indexed once, and each narrative node's description is enriched with retrieved context before local planning proceeds. This grounding step is what prevents contradictions across hundreds of scenes.
Visual consistency
Two complementary mechanisms address visual drift. Graph-based dependency tracking analyzes textual shot descriptions during planning to identify shared characters, environments, and objects; generation then follows a topological ordering in which independent branches execute in parallel while dependent shots use reference-conditioned generation conditioned on prerequisite visuals. Transition videos handle spatial coherence within scenes: rather than generating multiple viewpoints independently, the system generates a smooth camera transition between positions Ci and Cj, then extracts the individual views from it. The transition acts as a spatial anchor, exploiting the video model's internal geometric coherence to prevent contradictory 3D layouts—for example, maintaining consistent seating arrangement and furniture placement across reverse-angle dialogue shots.
Evaluation
The evaluation spans six research questions: overall performance, human preference, ablations, efficiency, LLM backbone sensitivity, and hyperparameter sensitivity.
On ViMax-Bench, ViMax-Gemini-3-Pro-Veo 3.1 leads all nine consistency columns (cross-scene CC, intra-scene IC, global GC) under Medium, Long, and Overall story groups, raising overall GC from 0.527 (strongest baseline, HoloCine) to 0.557. Gains are largest on the Long group, where keyframe-anchored baselines degrade as scene boundaries accumulate—the paper attributes this to global dependency tracking rather than per-clip quality alone.
| Method |
Overall CC |
Overall IC |
Overall GC |
| Veo 3.1 |
0.4978 |
0.5074 |
0.5005 |
| Wan2.2 |
0.4257 |
0.4294 |
0.4269 |
| IC-LoRA+Wan2.2 |
0.5147 |
0.5460 |
0.5241 |
| StoryDiffusion+Wan2.2 |
0.5212 |
0.5401 |
0.5269 |
| HoloCine |
0.5098 |
0.5686 |
0.5272 |
| ViMax-Gemini-3-Pro-Veo 3.1 |
0.5500 |
0.5772 |
0.5571 |
On novel-to-video adaptation over 50 NarrativeQA novels, ViMax improves average storyboard score from 3.83 to 4.11 and narrative faithfulness from 3.08 to 3.52 over Animaker, leading on six of seven dimensions. Notably, removing the event-aware process chain slightly improves visual specificity (4.94 → 4.98) but collapses narrative faithfulness from 3.52 to 2.34—a strong indication that explicit event-level causal structure, not richer shot descriptions, drives source fidelity.
Human evaluation
A pairwise study on all 35 benchmark cases (175 pairs, 525 judgments, three evaluators per pair) reports win rates against five baselines. ViMax wins 77.62% against StoryDiffusion and 73.33% against IC-LoRA overall, with its weakest margin against HoloCine (60.48%), whose intra-scene consistency remains competitive.
Ablations and efficiency
Removing all cross-shot dependencies (fully parallel generation conditioned only on character portraits) causes the largest degradation—an 8.7% drop in overall GC—establishing that portraits alone cannot sustain long-range coherence. Replacing structured camera-graph conditioning with naive first-frame chaining degrades eight of nine metrics. Removing VLM candidate selection lowers GC by 3.8%. On efficiency, the dependency-aware schedule dominates both extremes: strict serial generation is slowest and accumulates reference-image degradation, while full parallelism is fastest but most inconsistent; the graph schedule parallelizes independent branches and is faster than serial for long stories.
Backbone and hyperparameters
With the pipeline fixed, GPT-5.4 yields the best trade-off (GC 0.5623, Avg. 4.11); Claude-Sonnet-4.6 leads several narrative-quality dimensions but shows lower consistency scores, supporting the claim that local storyboard quality does not guarantee stable cross-shot visual state. For best-of-k selection at the keyframe stage, k=2 is optimal; k=3 and k=4 yield no gains, attributed to selection noise among plausible candidates.
Limitations and open questions
The paper concedes several constraints directly. The system depends on proprietary foundation models throughout, making results sensitive to upstream model updates—a concern reinforced by the observation that recent commercial models exhibit encouraging short-horizon multi-shot consistency natively, raising the question of whether harnesses like ViMax should shift toward long-horizon memory management rather than re-implementing capabilities base models may absorb. Automatic metrics are representation-similarity based and struggle with fine-grained intent, multi-person interaction artifacts, and director-level cinematic logic; ViMax-Bench itself is modest in scale (35 stories) and generated by an LLM. Generation components remain brittle: keyframe synthesis falters in crowded, contact-rich scenes, and the video generator does not always respect keyframe semantics (e.g., animating a treadmill-running character off the treadmill), with errors cascading downstream. Audio, dialogue synchronization, shot-level pacing, and interactive revision are unaddressed. An additional methodological caveat: the narrative-planning protocol compares baselines on their first 20 sub-storyboards against five consecutive 20-storyboard batches from ViMax, a matched-unit design that still advantages end-to-end long-horizon planners.
Conclusion
ViMax demonstrates that coordinated multi-agent orchestration—hierarchical decomposition grounded by RAG, dependency-graph scheduling with reference-conditioned generation, transition-video spatial anchoring, and VLM-based best-of-k filtering—yields measurable improvements in both cross-shot visual consistency and long-horizon narrative faithfulness over shot-level, keyframe-anchored, and unified multi-shot baselines. The ablations isolate cross-shot visual references as the single most critical component, and the backbone study indicates that stronger textual reasoning translates directly into better long-range state tracking. The framework's dependence on proprietary models and similarity-based evaluation leaves open how these gains will hold as foundation models internalize more multi-shot capability themselves.