Multi-Video Structured Prompt

• Multi-Video Structured Prompt is a framework that structures multiple video units into explicit segments by preserving temporal order, semantic roles, and graph relationships.
• It employs techniques like mask-guided key/value sharing, temporal decomposition, and graph fusion to ensure smooth transitions and coherent multi-video reasoning.
• Empirical findings across studies demonstrate that structured prompting improves continuity, computational efficiency, and control compared to naive concatenation of video tokens.

Searching arXiv for the cited works to ground the article in current literature. Multi-Video Structured Prompt is a family of prompting and conditioning paradigms in which multiple temporally or semantically related video units are not treated as an undifferentiated token stream, but as a structured object with explicit internal organization. Across recent work, the term covers several distinct but converging settings: temporally ordered prompt scripts for long-form video generation, graph-structured prompting for multi-video reasoning, structured text supervision over adjacent action clips, prompt banks for multi-shot extrapolation, and video-conditioned prompting via reference clips. The common premise is that naive concatenation—whether of prompts, video tokens, or clip descriptions—fails to preserve coherence, semantic alignment, or computational efficiency, whereas structured decomposition into segments, roles, graphs, or prompt fields enables better continuity, reasoning, and control (Cai et al., 2024).

1. Conceptual scope and formalization

In video generation, Multi-Video Structured Prompt is often framed as a temporal composition problem rather than a simple multi-text conditioning problem. DiTCtrl formulates the setting as: given a pretrained single-prompt text-to-video diffusion model $\mathcal{F}$ and an ordered prompt sequence $\{P_1, P_2, \dots, P_n\}$, generate a coherent long video that follows these prompts over time. The paper writes this as

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$

Here, each prompt governs a semantic phase or segment, and the central requirement is not simultaneous blending but temporally ordered realization with continuity of subject identity, motion, and scene evolution across segment boundaries (Cai et al., 2024).

In video language modeling, the same general idea appears in a different form. The early version of the structured multi-video collaborative reasoning framework takes as input a target video $V_0$ and related videos $\{V_1, V_2, \dots, V_N\}$, and argues that direct concatenation of all raw video tokens is counterproductive. Instead, it converts each video into a spatio-temporal graph, fuses target and related graphs, and serializes the result as a structured prompt containing target-video visual tokens, target-video graph tokens, related-video graph tokens, the question, and textual guidance, with the fixed Chain-of-Thought cue “Think through the process step by step” (He et al., 16 Sep 2025).

Bridge-Prompt provides an earlier precursor in instructional video understanding. Although it operates within one long instructional video rather than across separate videos, it explicitly converts a local sequence of adjacent actions into a hierarchy of text prompts: statistical, ordinal, semantic, and integrated. The integrated prompt

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$

is a textual serialization of a local action sequence and shows that structured prompting can encode order, count, and composition rather than only clip-local labels (Li et al., 2022).

Across these formulations, Multi-Video Structured Prompt denotes a shift from flat prompt strings or flat token concatenation toward prompt structures that preserve relations among video units: temporal order, semantic role, graph connectivity, narrative dependency, or prompt-source identity. This suggests a unifying interpretation: the “prompt” is no longer merely text, but an organized interface between multiple video-conditioned evidence sources and a downstream generator or reasoner.

2. Temporal composition for long-form video generation

DiTCtrl is one of the clearest formulations of structured prompting for generation under MM-DiT architectures. It argues that multi-prompt video generation should be treated as temporal video editing with smooth transitions rather than naive prompt concatenation. The method is training-free: no finetuning, retraining, or optimization is performed for the multi-prompt task; all control is injected at inference time by modifying the pretrained MM-DiT attention behavior and latent composition (Cai et al., 2024).

The architectural observation behind DiTCtrl is that MM-DiT full attention over concatenated text and video tokens can be decomposed into four interpretable regions: video-to-video, text-to-text, text-to-video, and video-to-text. The text-to-video and video-to-text regions behave similarly to cross-attention maps in UNet-based diffusion models, while video-to-video behaves like spatiotemporal self-attention. This makes training-free attention control possible in a Sora-like text-to-video model, concretely implemented on CogVideoX-2B (Cai et al., 2024).

Its core control mechanism is mask-guided key/value sharing across adjacent prompt segments. At denoising step $t$, DiTCtrl extracts cross-modal regional attention maps,

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$

derives prompt-specific masks $M_{i-1}$ and $M_i$, and then uses current-segment queries to attend to previous-segment keys and values separately for object and background regions: $\{P_1, P_2, \dots, P_n\}$0

$\{P_1, P_2, \dots, P_n\}$1

This enforces object-to-object and background-to-background transfer, preserving appearance consistency while still allowing semantic change (Cai et al., 2024).

To smooth boundaries visually, DiTCtrl uses overlapped latent windows between adjacent segments. If $\{P_1, P_2, \dots, P_n\}$2 and $\{P_1, P_2, \dots, P_n\}$3 are adjacent generated clips and $\{P_1, P_2, \dots, P_n\}$4 frames overlap, the latent at overlap position $\{P_1, P_2, \dots, P_n\}$5 is blended as

$\{P_1, P_2, \dots, P_n\}$6

with symmetric triangular weight

$\{P_1, P_2, \dots, P_n\}$7

The overlap thus becomes an explicit transition window between prompt-defined phases (Cai et al., 2024).

The experimental evidence is centered on MPVBench, a benchmark containing 130 long-form prompt sequences spanning 10 transition modes. The paper introduces CSCV, the Clip Similarity Coefficient of Variation, defined using adjacent-frame similarities

$\{P_1, P_2, \dots, P_n\}$8

and score

$\{P_1, P_2, \dots, P_n\}$9

On MPVBench, DiTCtrl achieves the best CSCV at 84.90%, compared with 84.37% for FreeNoise, 78.74% for FreeNoise+DiT, 74.97% for Video-Infinity, and 59.44% for Gen-L-Video. It also achieves the best motion smoothness score at 97.80%, though its text-image similarity is 30.68%, lower than FreeNoise at 32.69% and Video-Infinity at 32.35%, which the paper interprets as a continuity–prompt-adherence tradeoff caused by KV-sharing (Cai et al., 2024).

The paper’s broader implication is explicit: structured prompt sequences should be temporally ordered and locally compatible. Adjacent prompts should differ in controlled ways—action, style, camera behavior, or location—while preserving a specific consistent object or visual thread when possible. This makes “multi-video structured prompt” synonymous, in this line of work, with inference-time temporal composition over prompt-defined video segments (Cai et al., 2024).

3. Structured prompts for multi-video reasoning and retrieval

The multi-video collaborative reasoning framework for Video LLMs addresses a different failure mode: spatio-temporal incompleteness in individual videos. Its central claim is that simply feeding more raw videos into an LLM is harmful. On InternVid-QA, the “multi-video tokens” baseline inflates context length from 2.1K to 12.5K tokens and drops accuracy from 45.6 to 28.5. By contrast, the proposed graph-fusion prompt reaches 49.5 accuracy with only 2.3K tokens (He et al., 16 Sep 2025).

The foundation of this framework is the Video Structuring Module. Each video is converted into a spatio-temporal graph by scene detection, dense captioning, scene-graph parsing, grounding, and tracking. Textual scene graphs are represented as triplets

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$0

or $\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$1. Grounding produces

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$2

and tracking produces

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$3

Each grounded object region is cropped and encoded with OpenCLIP, using only the $\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$4 token as the node feature (He et al., 16 Sep 2025).

The Graph Fusion Module then performs two operations. First, a Hierarchical Frame Graph Attention Network propagates intra-video spatio-temporal structure through graph edges. Second, Cross-Graph Attention lets target-video nodes absorb useful information from related videos. With target features $\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$5 and related features $\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$6, the target query is

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$7

while keys and values come from concatenated target and related features: $\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$8

$\mathcal{V}_{\{1,...,n\} = \mathrm{DiTCtrl}\{\mathcal{F}(P_1),...,\mathcal{F}(P_n)\}.$9

A learnable class embedding distinguishes target and related nodes: $V_0$0 These class embeddings are added to attention inputs before fusion (He et al., 16 Sep 2025).

The structured prompt sent to the LLM interleaves target-video visual tokens, target-video graph tokens, related-video graph tokens, task instructions, the user question, and the fixed CoT cue. Crucially, only the target video retains raw visual tokens; related videos are represented only by graph tokens. This asymmetry is the central prompt-efficiency strategy (He et al., 16 Sep 2025).

Empirically, the framework is implemented on top of Video-LLaVA with Vicuna-7B v1.5 and achieves 61.8 accuracy on MSRVTT-QA, 46.9 on ActivityNet-QA, and 49.5 on InternVid-QA, improving over Video-LLaVA by $V_0$1, $V_0$2, and $V_0$3, respectively. Ablations show that removing graph structure or cross-graph fusion harms performance: direct projection of graph features yields 46.2 on InternVid-QA, adding HF-GAT lifts it to 47.8, and adding CGA yields 49.5, while adding an FFN afterward reduces accuracy to 48.7 (He et al., 16 Sep 2025).

A related but retrieval-oriented formulation appears in MAVIS, which rethinks text-to-video retrieval as structured query execution over a corpus-level Structured Semantic Library. Each video is parsed into a concise caption and a tuple

$V_0$4

with sub-libraries indexed as

$V_0$5

A planner decomposes a query into active semantic dimensions,

$V_0$6

specialized agents retrieve candidates, and a Logic-aware Debate applies a strict veto protocol before final visual verification (Zhang et al., 8 Jun 2026).

The debate mechanism is formalized with agent proposal sets

$V_0$7

pool

$V_0$8

and controversial set

$V_0$9

On MSR-VTT, MSVD, and ActivityNet, MAVIS achieves 78.6/94.9/96.3, 78.1/94.25/97.33, and 69.15/92.4/96.8 for R@1/R@5/R@10, respectively, without task-specific finetuning (Zhang et al., 8 Jun 2026).

Together, these works establish a general principle: when multiple videos are involved, structured prompts outperform raw multi-video token concatenation because they compress evidence into semantically typed units before LLM reasoning.

4. Sequence- and task-structured prompting in video representation learning

Bridge-Prompt shows that structured prompting can also serve as supervision rather than merely inference-time control. It reformulates adjacent action labels in instructional videos into a “three-plus-one-level” prompt engineering scheme. The statistical prompt is

“this video clip contains \underline{{num(K)} actions in total” and is denoted $\{V_1, V_2, \dots, V_N\}$0. The ordinal prompt is “this is the \underline{{ord_i} action in the video” and the semantic prompt is “\underline{{ord_i}, the person is performing the action step of \underline{{vp_i}” with integrated prompt given by

$\{V_1, V_2, \dots, V_N\}$1

The model then aligns these prompt levels with video cuts containing $\{V_1, V_2, \dots, V_N\}$2 consecutive actions through contrastive losses (Li et al., 2022).

Its feature extraction is based on a frame-wise encoder $\{V_1, V_2, \dots, V_N\}$3, a text encoder $\{V_1, V_2, \dots, V_N\}$4, and a fusion encoder $\{V_1, V_2, \dots, V_N\}$5. For the $\{V_1, V_2, \dots, V_N\}$6-th action, the clip representation is

$\{V_1, V_2, \dots, V_N\}$7

and similarity is computed by cosine similarity

$\{V_1, V_2, \dots, V_N\}$8

The total loss combines semantic, integrated, and statistical alignment terms (Li et al., 2022).

Empirically, Bridge-Prompt improves action segmentation and long-term activity recognition. On GTEA with ASFormer, it achieves F1@10 94.1, F1@25 92.0, F1@50 83.0, Edit 91.6, and Acc 81.2. On 50Salads, it achieves F1@10 89.2, F1@25 87.8, F1@50 81.3, Edit 83.8, and Acc 88.1. On Breakfast, Bridge-Prompt reaches 80.00% accuracy for long-term activity recognition (Li et al., 2022). The loss ablation shows that adding integrated and statistical prompt supervision materially improves performance beyond semantic prompts alone (Li et al., 2022).

A different form of structured prompting appears in PromptonomyViT, where prompts encode task structure rather than natural-language sequence structure. A fixed set of learned task prompt vectors

$\{V_1, V_2, \dots, V_N\}$9

is appended to the transformer token sequence: $$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$0 Each prompt corresponds to a synthetic scene-level task such as depth, normals, semantic segmentation, 3D pose, or boxes, and these prompt streams interact with video patch tokens through the entire transformer (Herzig et al., 2022).

Predictions are made from the downstream CLS token,

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$1

and from task-specific prompt tokens,

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$2

The training objective combines downstream classification with synthetic task losses: $$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$3 On SomethingElse compositional recognition, PViT reaches 65.5 top-1 / 89.0 top-5 versus 63.3 / 87.5 for MViTv2; on Ego4D object state change classification, it improves top-1 from 71.6 to 74.8; on Diving48 it improves from 73.1 to 85.8; and on AVA it improves mAP from 26.8 to 28.4 (Herzig et al., 2022).

DLM-VMTL extends the idea to heterogeneous video multi-task prompt learning. For each auxiliary task, cross-task prompt extraction is performed by

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$4

followed by a dimension corresponding unit

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$5

adapter-based alignment

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$6

concatenation

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$7

and injection into the primary task model: $$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$8 Across six tasks, DLM-VMTL surpasses full finetuning while using 10.8% of total parameters: for example, on SSv2 it reaches 76.7 versus 75.2 for full finetuning, on AVA 42.2 versus 40.4, and on YouTube-VIS 2019 66.1 versus 64.9 (Bo et al., 2024).

These works collectively broaden Multi-Video Structured Prompt beyond prompt strings. In representation learning, the prompt can be a structured sequence supervision signal, a bank of task tokens, or a task-organized transfer interface.

5. Prompt banks, recursive context, and multi-shot narrative generation

Later work makes the structure increasingly explicit at the shot level. PACR-Video formulates multi-shot long video extrapolation as sequential generation of ordered shots

$$y_{integ} = y^1_{sem} \oplus y^2_{sem} \oplus \cdots \oplus y^K_{sem}$$9

conditioned on shot-level prompts

$t$0

At step $t$1, the model conditions on

$t$2

where $t$3 is the current shot text encoding, $t$4 is routed historical prompt context, and $t$5 is a learned shot-role embedding (Córdoba et al., 7 Jul 2026).

The persistent memory is a recursive prompt bank: $t$6 storing compact entity, location, action, and style prompts for each prior shot. Narrative dependency routing is defined by

$t$7

The resulting routed context drives low-rank temporal adapters inserted into a frozen text-to-video diffusion transformer: $t$8 Training combines diffusion loss, identity contrast, and routing sparsity: $t$9 PACR-Video tunes only 3.8% of backbone parameters and improves over ReCA from FVD 268.4 to 231.7, CLIPScore 31.2 to 32.8, DINO identity consistency 0.724 to 0.771, and transition coherence 0.681 to 0.734, with 63.8% human preference over ReCA (Córdoba et al., 7 Jul 2026).

CausalCine addresses the same narrative problem from an autoregressive perspective. A multi-shot video with chunked causal factorization is modeled as

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$0

where chunk-level prompts are inherited from shot prompts via $${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$1. Training uses a packed clean/noisy teacher-forcing layout

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$2

with causal masking and tuning loss

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$3

Its Content-Aware Memory Routing stores key descriptors

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$4

scores relevance with

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$5

and defines the receptive field as

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$6

With $${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$7 chunks and $${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$8 frames, CausalCine reaches Aesthetic 0.6261, Text 0.1980, Subject 0.9717, Background 0.9675, and shot-cut accuracy 0.9732, outperforming autoregressive baselines and approaching bidirectional multi-shot systems while supporting prompt updates during rollout (Meng et al., 12 May 2026).

These shot-level systems make explicit what earlier multi-prompt work implied: a Multi-Video Structured Prompt can be represented as an ordered set of local prompts plus a story-global memory of reusable prompt fragments and role tokens.

6. Video-as-prompt, object-level prompting, and prompt refinement

Another branch of the literature expands the meaning of prompt beyond text. Video-As-Prompt reframes semantic control in video generation as in-context generation from a reference video rather than from a task-specific control map. It defines a unified semantic condition space

$${A^c_i} \in \mathbb{R}^{F\times H\times W \times N},$$9

over concept, style, motion, and camera, and conditions a single model on

$M_{i-1}$0

Its in-context token layout is

$M_{i-1}$1

and it trains under a flow-matching objective

$M_{i-1}$2

$M_{i-1}$3

$M_{i-1}$4

A temporally biased RoPE shifts the reference prompt’s temporal indices by an offset $M_{i-1}$5 to avoid false pixel-level mapping priors. On the benchmark, VAP achieves CLIP Score 24.13, Motion Smoothness 98.59, Dynamic Degree 77.08, Aesthetic Quality 57.71, Semantic Alignment 70.44, and User Preference 38.7%, with zero-shot transfer to unseen semantics such as crumble, dissolve, levitate, and melt (Bian et al., 23 Oct 2025).

VoCap applies prompt structure at the object level. Given a video

$M_{i-1}$6

and a prompt—text, box, or mask—it outputs a binary masklet

$M_{i-1}$7

and a caption string $M_{i-1}$8 for the corresponding object. Segmentation uses prompt-conditioned cross-attention: $M_{i-1}$9

$M_i$0

while captioning uses caption tokens

$M_i$1

followed by autoregressive decoding

$M_i$2

On SAV-Caption-val, VoCap reaches 47.8 CIDEr versus 35.5 for SAM2$M_i$3PixelLLM and 40.5 for SAM2$M_i$4Gemini pseudo-labeling, while preserving strong segmentation quality at 75.5 J&F (Uijlings et al., 29 Aug 2025).

Prompt refinement appears in both single-video and multi-path reasoning settings. Prompt-A-Video learns a model-specific prompt rewriting policy for text-to-video via reward-guided prompt evolution, SFT, and DPO. Its SFT objective is

$M_i$5

and DPO objective is

$M_i$6

On Open-Sora 1.2, average WebVid score improves from 3.116 with original prompts to 3.318 after two DPO rounds; on CogVideoX it improves from 2.989 to 3.056 (Gonzalez et al., 2024).

SCMAPR generalizes refinement into a multi-agent, scenario-aware, self-correcting pipeline. It routes prompts into one of 11 tags, synthesizes a scenario-specific policy, rewrites under that policy, atomizes the original prompt into a five-field dictionary

$M_i$7

containing characters, objects, actions, locations, and scenery, flattens the atoms

$M_i$8

chunks the rewritten prompt

$M_i$9

matches atoms to evidence with

$\{P_1, P_2, \dots, P_n\}$00

and validates each atom-evidence pair: $\{P_1, P_2, \dots, P_n\}$01 Coverage and contradiction rates are

$\{P_1, P_2, \dots, P_n\}$02

$\{P_1, P_2, \dots, P_n\}$03

A rewritten prompt is accepted only if $\{P_1, P_2, \dots, P_n\}$04 and $\{P_1, P_2, \dots, P_n\}$05. On VBench, SCMAPR improves Wan from 86.19 to 88.21 and LaVie from 81.89 to 84.56; on EvalCrafter, Wan improves from 63.46 to 66.74 (Yang et al., 7 Apr 2026).

Team of One, although evaluated on single-video QA, contributes a prompt-ensemble perspective that is structurally compatible with multi-video prompting: multiple reasoning pathways—contextual semantics, action-existence verification, temporal/causal reasoning, and question-driven analysis—produce multiple candidate answers, and an external evaluator selects or fuses them (Xie et al., 18 Jul 2025). This suggests that structured prompting need not only organize inputs; it can also organize parallel reasoning paths.

7. Limitations, misconceptions, and design implications

A recurrent misconception is that Multi-Video Structured Prompt simply means concatenating multiple prompts or multiple videos. The literature surveyed here repeatedly rejects that view. DiTCtrl explicitly argues that multiple sequential prompts should not be blended simultaneously but realized in temporal order with continuity constraints (Cai et al., 2024). The collaborative reasoning framework shows that direct multi-video token concatenation can sharply degrade performance and inflate context length (He et al., 16 Sep 2025). MAVIS similarly argues that brute-force full-corpus embedding comparison is semantically mismatched to sparse queries (Zhang et al., 8 Jun 2026).

A second misconception is that more structure always means symbolic rigidity. Several systems instead use soft, learned, or hybrid structures. DiTCtrl uses masks from cross-modal attention rather than symbolic scene annotations (Cai et al., 2024). PACR-Video stores prompt-bank vectors rather than explicit graphs (Córdoba et al., 7 Jul 2026). Video-As-Prompt uses a reference video as semantic context rather than task-specific control maps (Bian et al., 23 Oct 2025). PromptonomyViT’s prompts are learned task tokens rather than human-readable instructions (Herzig et al., 2022).

The main limitations also recur across works. Structured multi-video reasoning frameworks depend on retrieval quality and upstream graph or caption parsing; less relevant auxiliary videos reduce accuracy (He et al., 16 Sep 2025). Structured prompt banks can drift over long horizons, and dependency prediction errors can route stale or irrelevant context (Córdoba et al., 7 Jul 2026). Multi-reference video prompting can cause semantic or appearance leakage when prompt videos are insufficiently disambiguated (Bian et al., 23 Oct 2025). Prompt refinement methods remain largely single-video unless augmented with cross-video consistency rewards or global continuity constraints (Gonzalez et al., 2024). PPLLaVA’s prompt-guided compression is highly relevant to multi-video settings, but its support for true multi-video reasoning remains implicit rather than directly benchmarked (Liu et al., 2024).

Several robust design principles nevertheless emerge.

First, local structure should be explicit. Whether the unit is an action clip, a shot, a graph node, an object track, or a prompt phase, systems benefit when they encode local roles rather than flattening all evidence. Second, global continuity should be stored separately from local intent. PACR-Video’s distinction between current shot prompt and recursive prompt bank is a particularly clear example (Córdoba et al., 7 Jul 2026). Third, prompt or evidence selection should be query- or context-adaptive rather than uniform: this is the shared logic behind prompt-guided pooling, content-aware memory routing, graph fusion, and logic-aware debate (Liu et al., 2024, Meng et al., 12 May 2026, He et al., 16 Sep 2025, Zhang et al., 8 Jun 2026). Fourth, structured systems increasingly rely on verification loops. SCMAPR’s atom-level entailment checking is the most explicit instance, but DiTCtrl’s mask-guided transfer and MAVIS’s veto protocol serve related roles in generation and retrieval (Yang et al., 7 Apr 2026, Zhang et al., 8 Jun 2026). Fifth, prompt representations need not be textual only. In current practice, prompts may be temporal scripts, learned task tokens, object prompts, graph tokens, auxiliary modality streams, or reference videos (Cai et al., 2024, Herzig et al., 2022, Uijlings et al., 29 Aug 2025, Guo et al., 2024, Bian et al., 23 Oct 2025).

Viewed across these lines of research, Multi-Video Structured Prompt is best understood not as a single method but as an organizing paradigm for video AI systems. It replaces flat conditioning with structured interfaces that reflect the compositional, temporal, and multi-source nature of video itself.