MEve: Multi-Event Video Generation Benchmark

• MEve is a prompt-based benchmark for multi-event text-to-video generation, using explicit event segmentation and temporal order to guide diffusion models.
• It employs dual-event and four-event prompt formulations to diagnose when and where event transitions occur during the denoising process.
• MEve reveals key failure modes such as event fusion, scrambling order, and transition collapse, informing improvements in video synthesis methods.

MEve is a self-curated prompt suite and evaluation benchmark for multi-event text-to-video generation. It was introduced to study when and where temporally ordered events are controlled during diffusion-based video synthesis, and it was later used as the primary benchmark for training-free progressive steering of diffusion transformers. In the associated papers, MEve is a prompt-based benchmark rather than a conventional video dataset, with explicit event segmentation, temporal order, and viewpoint control; one formulation centers on dual-event prompts for diagnostic probing, while another extends the benchmark to multi-event video generation up to 4 events (Liao et al., 3 Oct 2025, Liao et al., 29 May 2026).

1. Concept and scope

MEve is defined as a benchmark prompt suite for multi-event video generation. In the dual-event study, it is described as a self-curated prompt suite and evaluation benchmark specifically designed to probe multi-event text-to-video generation and to answer the question of when and where multi-event prompts control event transitions during diffusion generation (Liao et al., 3 Oct 2025). In the later TunerDiT work, it is introduced as “a benchmark prompt suite for multi-event video generation up to 4 events in multiple aspects,” and is used as the primary benchmark for evaluating long-horizon generation with multiple sequential events (Liao et al., 29 May 2026).

A central design choice is that MEve is prompt-centric. The benchmark provides textual instructions that describe sequentially ordered events, and models generate videos conditioned on those prompts. This distinguishes MEve from benchmarks organized around fixed reference videos. The event unit is explicitly tied to a subject–predicate–object–time structure, which is used to enforce that each event is a relatively complete semantic unit rather than a loosely specified scene fragment (Liao et al., 29 May 2026).

Two related formulations are associated with the name. The first uses exactly two events per prompt in order to isolate event transitions and identify turning points in denoising steps and model depth (Liao et al., 3 Oct 2025). The second organizes the benchmark around 4-event prompts, while evaluating 2-, 3-, and 4-event generation regimes by slicing or scheduling those prompts (Liao et al., 29 May 2026). This suggests that MEve functions both as a controlled diagnostic scaffold and as a broader stress test for long-horizon multi-event synthesis.

2. Motivation and relation to prior benchmarks

MEve was created in response to a gap in existing text-to-video evaluation practice. The benchmark papers state that VBench, VBench++, and VBench 2.0 focus on single-shot generation, dual-shot transitions, and seldom tackle multi-event prompts systematically, while MPVBench emphasizes transition effects between two similar scenes and deliberately avoids logical sequential prompts (Liao et al., 29 May 2026). The dual-event MEve paper makes the same critique in procedural terms: most benchmarks measure global text–video alignment or overall realism, but do not explicitly test sequential events or analyze how the generative process switches from one event to another (Liao et al., 3 Oct 2025).

The distinction between multi-scene and multi-event is also important. The later paper notes that WorldScore mixes multi-scene and multi-event notions and does not explicitly adhere to an event definition grounded in subject–predicate–object–time (Liao et al., 29 May 2026). MEve therefore targets event order, event separation, transitions, identity persistence, and viewpoint control under explicit temporal sequencing.

The benchmark is also motivated by observed failure modes of naive prompting. In the dual-event formulation, simply conditioning a model on a prompt of the form $$\mathbf{P} = [\,\text{P}_1 \;\text{then}\; \text{P}_2\,]$$ often yields mixed or blended scenes, neglect of $\text{P}_2$, or unnatural transitions (Liao et al., 3 Oct 2025). In the later study, the same family of failures is named more explicitly as event fusion, scrambling order, and transition collapse (Liao et al., 29 May 2026). MEve was designed to make such failures measurable rather than anecdotal.

3. Construction and benchmark composition

Across the related papers, MEve keeps a stable category inventory while varying the number of events per prompt. The categories are General, Motion Order, Human Identity, Complex Plot, and Ego–Exo4D. In both descriptions, the prompt counts are 60, 98, 32, 60, and $50 \times 2$, respectively, yielding 350 prompts in total (Liao et al., 3 Oct 2025, Liao et al., 29 May 2026).

Category	Source or diagnostic focus	# Prompts
General	Gemini 2.5 Pro; generic narratives	60
Motion Order	VBench 2.0 motion-order content	98
Human Identity	VBench 2.0 identity consistency content	32
Complex Plot	VBench 2.0 multi-step narrative content	60
Ego–Exo4D (paired)	First-/third-person variants from narrations	100

Prompt construction follows three main sources. First, the General prompts are authored with Gemini 2.5 Pro as a controlled prompt generator, producing narratives such as “event_1 then event_2 then …” while attempting to avoid strong priors or biases toward particular scenes (Liao et al., 29 May 2026). In the dual-event version, Gemini is instructed that each prompt should contain two events, the events should be easy to visually distinguish, and the causality of the events should be preserved (Liao et al., 3 Oct 2025).

Second, diagnostic prompts are derived from VBench 2.0 categories that are particularly relevant to multi-event generation: Motion Order Understanding, Human Identity, and Complex Plot. In the 4-event formulation, single-scene prompts from those categories are converted into four sequential events while preserving their diagnostic intent (Liao et al., 29 May 2026). In the dual-event formulation, the same categories are adapted into two-event prompts; for Complex Plot, Gemini 2.5 is used to decompose complex narratives into two event parts, and pronouns in the second event are replaced by explicit subjects from the first event to reduce ambiguity (Liao et al., 3 Oct 2025).

Third, MEve incorporates real-world event sequences from Ego-Exo4D narrations. These are transformed into paired egocentric and exocentric prompts for viewpoint control. The first-person variant prepends “Generate a first-person view video …” and rewrites the subject as “the camera wearer,” while the third-person variant prepends “Generate a third-person view video …” and rewrites the subject as “the person” (Liao et al., 29 May 2026). This gives MEve explicit view control in addition to multi-event structure.

4. Evaluation protocol and metrics

MEve is evaluated in zero-shot settings: models are not fine-tuned on the benchmark, but are tested directly under controlled conditioning schedules. In the dual-event study, CogVideoX-5B, CogVideo1.5X-5B, OpenSora 1.2, and OpenSora 2.0 are evaluated using each model’s native frame count and resolution (Liao et al., 3 Oct 2025). In the later study, models again generate videos with their native supported frame counts and resolutions under identical MEve prompts, and evaluation is organized for 2-, 3-, and 4-event settings (Liao et al., 29 May 2026).

A defining feature of MEve is dynamic conditioning. For temporal switching, the dual-event paper defines

$$\mathrm{cond}(\tau) = \begin{cases} \text{P}_1, & \tau < x,\ \text{P}_2, & \tau \ge x, \end{cases}$$

with switch index $k = \lfloor xN \rfloor$ for $N$ denoising steps, and sweeps $x \in \{0, 0.1, 0.2, \dots, 1.0\}$ to probe when event control changes (Liao et al., 3 Oct 2025). For block-wise switching in OpenSora 1.2, the same paper uses $b = \lfloor xB \rfloor$, conditioning shallow DiT blocks on $\text{P}_1$ and deeper blocks on $\text{P}_2$ to probe where event control changes in model depth (Liao et al., 3 Oct 2025).

The metric suite is correspondingly multi-layered. In the dual-event study, Text Alignment (TA) uses ViCLIP features to measure alignment between event-specific text and event-specific video segments, Identity Consistency (IC) uses YOLOv5 person detections and DINOv2 embeddings across event mid-frames, and Background Consistency (BC) uses SAM2 background segmentation with DINOv2 embeddings (Liao et al., 3 Oct 2025). In the later study, automatic metrics include TA, Text–Image Similarity (TIS), BC, IC, and Clip Similarity Coefficient of Variation (CSCV), with event-level text–video features, background and foreground segmentation, and temporal smoothness measurements adapted to multi-event videos (Liao et al., 29 May 2026).

MEve also supports VLM-as-a-judge evaluation. The later paper samples frames from generated videos, uses Gemini-2.5-Flash to caption frames and summarize the clip, and then defines Event Isolation (EI) as event-count accuracy and Text–Video Alignment (TVA) as Sentence-BERT cosine similarity between the prompt and the summary caption (Liao et al., 29 May 2026). Human evaluation is added through 5-point Likert scoring on overall preference, motion naturalness, transition smoothness across events, and text–video alignment (Liao et al., 29 May 2026).

5. MEve as a diagnostic instrument for diffusion transformers

MEve was designed not only to rank systems but also to expose the internal control structure of diffusion-based video generation. In the dual-event analysis, MEve is the scaffold for asking when and where event switches occur. By sweeping the temporal switch ratio $\text{P}_2$0, measuring event-specific TA curves, and observing their intersection, the study defines turning points at which the dominant generated event shifts from $\text{P}_2$1 to $\text{P}_2$2 (Liao et al., 3 Oct 2025). The same benchmark is then used to examine block-wise switching inside DiT layers.

The resulting picture is highly specific. Early denoising steps, approximately the first 0–30% of the schedule, are found to dominate high-level content and event choice; introducing $\text{P}_2$3 only at later steps rarely produces a clean second event (Liao et al., 3 Oct 2025). Shallow layers in the DiT are likewise found to govern global spatio-temporal layout and event identity, whereas deeper layers are more limited to appearance refinement and local detail (Liao et al., 3 Oct 2025). The later TunerDiT paper states the same phenomenon in different terms, describing intrinsic turning points in the denoising trajectory where conditioning text affects generation from global layout to fine-grained details (Liao et al., 29 May 2026).

This mechanistic role of MEve directly shapes method design. TunerDiT builds on the turning-point analysis by introducing Event-Partitioned Masking, which enforces event boundaries while allowing cross-event transition bands, and Cross-Event Prompt Fusion, which injects neighboring event semantics for late-stage refinement (Liao et al., 29 May 2026). MEve is then used to tune the trade-off between event separation and video consistency, making it a benchmark for both analysis and intervention.

6. Findings, significance, and limitations

MEve has been used to reveal systematic failure modes in multi-event text-to-video generation. The later benchmark paper identifies event fusion, in which multiple events blend into a single scene; scrambling order, in which events appear in the wrong order or overlap; and transition collapse, in which boundaries become either abrupt frame-wise jumps or over-smoothed morphing without clear isolation (Liao et al., 29 May 2026). The dual-event study similarly shows that naive concatenation of prompts tends to favor the early part of the prompt and yields blended or ambiguous outputs (Liao et al., 3 Oct 2025).

On MEve, TunerDiT is reported to achieve state-of-the-art performance across 8 metrics and to offer a tunable trade-off between video consistency and event separation compared with other training-free methods (Liao et al., 29 May 2026). The benchmark is also used to show that the improvement in text alignment increases with event count, which the paper presents as indicating a scaling possibility with increasing event count (Liao et al., 29 May 2026). In the dual-event study, the main cross-model conclusion is architectural rather than leaderboard-oriented: early denoising steps and shallow layers remain decisive across both OpenSora and CogVideoX families (Liao et al., 3 Oct 2025).

MEve’s limitations are also explicit. The later work notes a token-budget constraint: under a typical 512-token text embedding window, increasing the number of events reduces the textual richness available per event (Liao et al., 29 May 2026). It also notes that MEve remains a curated prompt suite with limited domain coverage relative to real-world event diversity, and warns of “Consistency Hacking,” in which incorrect multi-event videos can artificially increase BC, IC, or CSCV by collapsing to static or fused scenes (Liao et al., 29 May 2026). These limitations do not negate the benchmark’s role; they delimit it. A plausible implication is that MEve is best understood as a high-control diagnostic benchmark for temporal event structure rather than a complete surrogate for open-domain video generation.

Taken together, the MEve line of work establishes a benchmark family for explicit multi-event prompting, event-order evaluation, and mechanistic probing of conditioning schedules in diffusion transformers. Its main contribution is not only the provision of prompts, but the formalization of multi-event video generation as a problem of event segmentation, temporal switching, viewpoint control, and denoising-stage sensitivity (Liao et al., 3 Oct 2025, Liao et al., 29 May 2026).