EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation

Abstract: Multi-shot video generation extends single-shot generation to coherent visual narratives, yet maintaining consistent characters, objects, and locations across shots remains a challenge over long sequences. Existing evaluations typically use independently generated prompt sets with limited entity coverage and simple consistency metrics, making standardized comparison difficult. We introduce EntityBench, a benchmark of 140 episodes (2,491 shots) derived from real narrative media, with explicit per-shot entity schedules tracking characters, objects, and locations simultaneously across easy / medium / hard tiers of up to 50 shots, 13 cross-shot characters, 8 cross-shot locations, 22 cross-shot objects, and recurrence gaps spanning up to 48 shots. It is paired with a three-pillar evaluation suite that disentangles intra-shot quality, prompt-following alignment, and cross-shot consistency, with a fidelity gate that admits only accurate entity appearances into cross-shot scoring. As a baseline, we propose EntityMem, a memory-augmented generation system that stores verified per-entity visual references in a persistent memory bank before generation begins. Experiments show that cross-shot entity consistency degrades sharply with recurrence distance in existing methods, and that explicit per-entity memory yields the highest character fidelity (Cohen's d = +2.33) and presence among methods evaluated. Code and data are available at https://github.com/Catherine-R-He/EntityBench/.

• The paper introduces EntityBench and demonstrates that explicit per-entity memory (EntityMem) significantly improves character consistency and prompt-following accuracy across long sequences.
• It proposes a three-pillar evaluation framework that rigorously assesses intra-shot quality, prompt alignment, and cross-shot consistency using 51 detailed metrics.
• Empirical results reveal that while EntityMem outperforms baselines on character fidelity and long-range robustness, challenges remain in object compositing and uniform embedding similarity.

Authoritative Summary of "EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation" (2605.15199)

Problem Definition and Benchmark Proposal

The paper addresses a crucial challenge in multi-shot video generation: maintaining visual consistency of entities (characters, objects, locations) across shots in long-form narratives. Existing benchmarks and evaluation protocols are limited in scope, typically focusing on single-shot quality, short sequences, or entity consistency for only one entity type, without fine-grained annotations or rigorous consistency metrics. The absence of a robust standard prevents systematic diagnosis and comparison of methods regarding entity consistency failures, especially for realistic, multi-entity narratives over dozens of shots.

To fill this gap, the authors introduce EntityBench, a multi-shot video generation benchmark derived from real narrative media and enriched for rigorous testing. EntityBench contains:

• 140 episodes, 2,491 shots, annotated with per-shot entity schedules specifying appearing characters, objects, and locations.
• Difficulty tiers (easy, medium, hard), extending up to 50 shots per episode and recurrence gaps up to 48 shots.
• Explicit annotation of cross-shot recurrence, scene transitions (cuts/continuations), and detailed entity regimens spanning up to 13 characters, 8 locations, and 22 objects per episode.
• Nearly 70% of entity appearances are memory-only, with no description at recurrence, directly probing long-range memory and identity preservation.

Evaluation Framework: Three-Pillar Metric Suite

The benchmark is paired with a rigorous, hierarchical evaluation suite comprising three pillars and fifty-one metrics:

• Pillar 1 (Intra-shot quality): Standard VBench dimensions (subject consistency, imaging quality, aesthetic quality, motion smoothness, etc.) rate technical quality independently per shot.
• Pillar 2 (Prompt-following alignment): Uses GroundingDINO, CLIP, and multimodal LLMs to measure presence and fidelity of scheduled entities, and correctness of action depiction.
  • Per-entity fidelity is scored via LLM against registry descriptions, with granular subcriteria (e.g., face/hair/clothing/build for characters).
• Pillar 3 (Cross-shot consistency): Evaluates whether recurring entities remain visually stable via embedding similarity (DINOv2 centroid), pairwise LLM identity judgments, and boundary metrics at scene transitions.
  • A fidelity gate ensures cross-shot metrics are evaluated only on correctly rendered entities, preventing artifacts from inflating scores.

Method: EntityMem—Explicit Entity Memory Management

The authors propose EntityMem, a memory-augmented multi-shot video generation system, building on prior persistent-memory work but extending it to explicit per-entity memory. Before video synthesis begins, VLM-based agents generate verified visual and textual references for each entity, populating a persistent memory bank:

• Characters receive segmented, labeled portraits; locations get panoramic backgrounds; mobile objects are separately classified.
• A Layout Agent plans shot compositions and camera angles using these references.
• During generation, the backbone retrieves per-entity guidance for every scheduled entity, maintaining identity independent from scene context and preventing error accumulation.

EntityMem operates without additional training, leveraging agentic context management and deterministic visual tools (segmentation, compositing).

Strong Empirical Results and Metric Analysis

EntityMem is evaluated head-to-head against three representative baselines:

• StoryMem: Persistent memory keyframe retrieval but lacking per-entity granularity.
• HoloCine: Holistic generation with joint denoising across all shots.
• CineTrans: Transition-focused cinematic shot boundary generation.

Key findings include:

• Character Fidelity: EntityMem achieves the highest character presence (0.967 vs. 0.849 for StoryMem) and prompt-following accuracy, with face_fidelity reaching 0.740 vs. 0.452 (+2.33 Cohen's d), and substantial improvements in hair, clothing, and build fidelity.
• Action Correctness: EntityMem leads on overall action correctness, subject_identity, and object_interaction metrics.
• Cross-shot Consistency: On LLM-judged identity metrics, EntityMem dominates (llm_face_accuracy 0.406 vs. 0.226), despite slightly lower embedding similarity scores (cs_face) relative to baseline. This exposes a trade-off: embedding similarity rewards uniformity but not accurate identity, while LLM metrics are more attuned to the preservation of detailed character traits.
• Long-range Robustness: EntityMem's advantage compounds with episode length; character-centric metrics remain robust as recurrence gaps increase, with minimal drop-off in LLM identity scores even at gaps >20 shots.
• Object Consistency: EntityMem lags baselines on cross-shot object metrics, attributed to compositing artifacts and lower backbone performance when integrating standalone object references.

Visual quality metrics (VBench) are orthogonal to entity consistency; some baselines achieve higher imaging and aesthetic scores but much lower entity fidelity and presence.

Implications and Theoretical Perspectives

EntityBench provides the first standardized setting for diagnosing entity inconsistency failure modes over realistic, multi-shot, multi-entity narratives. The fidelity gate and multi-pillar evaluation expose weaknesses in architectural designs that rely solely on scene-level conditioning or embedding sharing. The clear distinction between embedding-based and identity-based metrics suggests the need for protocols that optimize for human-recognizable identity rather than statistical visual uniformity.

EntityMem demonstrates that explicit per-entity memory management, informed from verified references and agentic layout planning, substantially improves identity preservation, especially for long-range, memory-only recalls. The gap-decay results reinforce the importance of such persistent memory, as traditional scene-level approaches degrade sharply with increasing recurrence gap.

Practically, these advances facilitate more reliable automated storytelling, animation prototyping, accessibility content, and educational media creation with consistent recurring characters.

Speculation and Future Directions

Future multi-shot T2V systems may incorporate more sophisticated memory architectures:

• Dynamic, adaptive memory banks that evolve during generation, recursively correcting drift.
• Active feedback loops from LLM-based evaluators for real-time correction.
• Unified pipelines for both character and object consistency, integrating compositional reasoning and cross-modal retrieval.

Benchmarks like EntityBench can catalyze architecturally novel systems capable of consistent narrative modeling. The distinction between embedding uniformity and identity preservation is likely to encourage development of new contrastive, agentic, or retrieval-based methods focused on long-range consistency.

Anticipated research directions include:

• Holistic multi-agent frameworks for full movie-level synthesis.
• Large-scale studies of human agreement and perceptual significance of entity drift.
• Safety, provenance, and attribution protocols for content generated with persistent entity memory.

Conclusion

"EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation" (2605.15199) establishes EntityBench as the most comprehensive benchmark for evaluating entity consistency in multi-shot video generation, and demonstrates the effectiveness of per-entity memory management (EntityMem) for improving long-range character fidelity. The rigorous metric suite and strong empirical findings offer a standardized platform for diagnosing and advancing methods in entity-consistent video generation, setting a new baseline for both practical applications and theoretical development in automated, narrative-driven content creation.