Keyframe-Anchored Identity Preservation for Sequential-Action Video Generation

Abstract: Identity-preserving text-to-video generation aims to synthesize a video that accurately follows a textual description while maintaining the recognizability of a user-specified subject throughout. The IPVG26 challenge extends this framework from a single holistic prompt to a temporally structured specification. The model additionally receives a sequence of timestamped action captions and must render the subject performing these actions in the specified order. This temporal structure presents a challenge not encountered in previous identity-preserving generation tasks, as the subject must continuously perform a scripted sequence of distinct actions while maintaining a consistent identity. However, end-to-end video generators are prone to appearance drift as motion accumulates and the depicted actions change. We address this challenge with a training-free, three-stage pipeline framework. An action-aware prompt polishment stage first rewrites the inputs into image-generation prompts that specify the terminal state of each action. An identity-preserving generation stage then produces the keyframe sequence by conditioning each frame jointly on the reference identity and its predecessor, thereby decoupling time-invariant appearance from time-varying pose. Finally, an identity-aware inference enhancement stage synthesizes the intermediate segments using multi-reference guidance and identity-driven noise searching, both of which reinforce identity fidelity during sampling. Our method ranked third on the official Track 2 leaderboard, demonstrating competitive performance and strong generality.

• The paper introduces a training-free three-stage pipeline that rewrites action prompts, generates chained identity-anchored keyframes, and interpolates video segments to reduce appearance drift during sequential actions.
• The method combines multi-reference guidance and noise searching, achieving an overall score of 0.4971 with identity, global alignment, and temporal alignment scores of 0.4055, 0.6293, and 0.5266, respectively.
• The approach ranked third in the IPVG26 Track 2 challenge, while ablations show that multi-reference guidance provides the largest identity improvement, raising the identity score by approximately 0.054 over the version without it.

This paper addresses Track 2 of the IPVG26 challenge, where a video generator must render a user-specified subject — given only a reference image — performing a scripted sequence of timestamped actions while remaining recognizable throughout. The authors' central observation is that end-to-end video diffusion models suffer appearance drift as motion accumulates and the depicted action changes, and that no prior identity-preserving generation method was designed for a temporally structured, multi-action specification. Their response is a training-free, three-stage pipeline that decomposes the task into a chain of identity-preserving keyframes anchored at action-segment endpoints, with the video synthesized by interpolating between consecutive keyframes. The method ranked third on the official Track 2 leaderboard, with gains concentrated on the identity-preservation and temporal-alignment metrics that carry the heaviest weights in the official scoring function.

Task setting and motivation

Track 2 provides a reference identity image, a global scene description, and a sequence of timestamped action captions $\{(s_i, e_i, c_i)\}$. A valid output must simultaneously preserve the reference identity, align with the global scene and each per-segment caption, and remain temporally coherent across segment boundaries. The authors argue this setting exposes a structural weakness of end-to-end reference-conditioned video generators: their joint optimization of appearance and motion induces a spatiotemporal trade-off, so identity degrades precisely when the subject transitions between distinct actions. Existing approaches — per-identity fine-tuning, offline-trained identity branches, and zero-shot reference-conditioned video models — are all built around a single holistic prompt rather than a scripted action timeline.

The keyframe-anchored decomposition sidesteps this trade-off by separating time-invariant appearance from time-varying pose: identity is enforced at discrete anchor points, and motion is recovered by interpolation. Because the entire pipeline operates on off-the-shelf models without fine-tuning, it inherits improvements in the underlying generators without retraining.

Method: three-stage pipeline

Action-aware prompt polishment

Video-level captions lack the spatial specificity image generators require, so a multimodal LLM (Qwen3.6-27B) performs a controlled two-stage rewrite. First, the global scene description is rewritten into a first-frame prompt that anchors identity attributes observed in the reference image (face shape, skin tone, hairstyle, distinctive facial traits) while establishing scene composition, framing, and lighting. Second, each temporal caption is independently rewritten into an end-frame prompt specifying the terminal state of its segment — body pose, hand position, and gaze direction at the moment the action concludes — while maintaining spatial continuity with the preceding frame. The emphasis on terminal states is functionally motivated: since the keyframe generator produces the ending frame of each segment, the prompt must describe where the action lands rather than how it unfolds. This design provides unambiguous spatial targets and promotes smooth inter-segment transitions.

ID-preserving chained keyframe generation

The first keyframe is generated from the reference image and the polished global prompt, depicting a neutral initial pose. Each subsequent keyframe is conditioned on three inputs simultaneously: the reference image as a persistent identity anchor, the preceding keyframe as a visual continuity cue, and the polished end-frame prompt as the target terminal state. The model is instructed to inherit pose, position, and orientation from the predecessor and modify only what the caption describes. This assigns the two orthogonal constraints to distinct inputs — the reference supplies time-invariant facial features, while the predecessor keyframe encodes the time-varying spatial state — so each transition reflects only the incremental change dictated by the corresponding action. Keyframes are synthesized by Z-Image at $1536 \times 1024$ resolution.

Video synthesis and identity-aware inference enhancement

Each segment is generated by a diffusion video model conditioned on its pair of boundary keyframes as first-frame and last-frame constraints. Because segments have unequal durations, a variable-length allocation strategy assigns the shortest segment a base budget of $F_{\min} = 121$ frames, with all other segments scaled proportionally and rounded to the nearest length of the form $8k+1$ to satisfy the temporal VAE constraint. Segments are concatenated at shared boundary frames, discarding the duplicated first frame of each non-initial segment.

Two training-free sampling-time enhancements reinforce identity fidelity. Multi-reference guidance (MRG) extends classifier-free guidance with a three-term formulation combining standard CFG, a spatiotemporal guidance term obtained by perturbing selected attention layers (transformer block 29), and a modality guidance term that nulls the image conditioning to amplify the identity-bearing boundary keyframes, with scales $w_{\text{cfg}} = 5.0$, $w_{\text{stg}} = 1.5$, and $w_{\text{mod}} = 3.0$. Noise searching (NS), adapted from ConsistTalk, treats the initial noise as a searchable variable: $K$ candidate noise tensors undergo short lookahead denoising, and the candidate whose partially decoded first frame maximizes cosine similarity between its face embedding and the reference is selected for full denoising. Both mechanisms convert stochastic sampling choices into identity-constrained decisions.

Experimental results

The evaluation covers the official Track 2 test set (200 samples built on ReactID), scoring identity preservation (CurricularFace/ArcFace for faces, DINOv2 for objects and animals), global and temporal text-video alignment (CLIP and GME), and visual quality (motion smoothness and image quality from VBench). The official aggregate weights identity at 0.20 and temporal GME at 0.25, making these the dominant dimensions.

Comparing two candidate backbones within the identical keyframe pipeline, LTX-2.3 in keyframe-interpolation mode outperformed Wan2.1-VACE in first-/last-frame mode on all seven metrics, raising the overall score from 0.4818 to 0.4971. Notably, motion smoothness is near saturation for both backbones (above 0.98), so the discriminative signal resides almost entirely in identity and alignment — precisely the dimensions the pipeline's design targets. On the official leaderboard the entry ranked third with a score of 2.9375, behind USTC-CMI (1.25) and WislabGDUT (1.8125), a gap the paper does not analyze in detail.

The ablation study isolates the contribution of the two inference-time enhancements:

Variant	ID	G-GME	T-GME	MS	IQ	Overall
Full model	0.4055	0.6293	0.5266	0.9883	0.6078	0.4971
w/o MRG	0.3520	0.6248	0.5176	0.9870	0.6011	0.4820
w/o NS	0.3853	0.6270	0.5223	0.9879	0.6052	0.4910

Removing MRG causes the largest degradation, dropping the identity score by roughly 0.054, confirming that explicitly amplifying the identity-bearing visual condition is the primary driver of subject consistency — text and keyframe conditioning alone do not sufficiently steer the sampling trajectory toward the reference appearance. Removing NS has a milder but consistent effect. Both components leave quality metrics nearly unchanged, indicating their benefits concentrate specifically on identity preservation. Qualitatively, the case study shows facial structure, skin tone, and hairstyle held consistent across four chained keyframes, with each segment converging on its caption's terminal state and segment boundaries free of abrupt jumps.

Limitations and open questions

The paper concedes that the noise search incurs $K$ additional short forward passes per segment, a nontrivial inference cost, though no latency figures are reported. The leaderboard margin to the top-ranked team is substantial (2.9375 versus 1.25), and the paper offers no error analysis of where its pipeline loses points relative to the winners. The evaluation is confined to a single 200-sample challenge test set, so generalization beyond this distribution is unverified. Several questions remain open: how the pipeline behaves when segment boundaries require large pose changes that chained keyframe conditioning cannot smoothly bridge; whether the fixed guidance scales ($w_{\text{cfg}}$, $1536 \times 1024$0, $1536 \times 1024$1) transfer across backbones; and whether the terminal-state prompt design degrades for actions whose intermediate dynamics, rather than endpoints, carry semantic meaning.

Conclusion

The paper demonstrates that a keyframe-anchored decomposition — chained identity-preserving keyframes at segment endpoints, terminal-state prompt rewriting, and identity-aware inference-time guidance and noise search — is an effective, fully training-free approach to identity-preserving multi-action video generation. Its third-place ranking, with ablations attributing the largest gains to multi-reference guidance on the most heavily weighted metrics, supports the claim that decoupling time-invariant appearance from time-varying pose mitigates the appearance drift that afflicts end-to-end generation under sequential-action specifications.