X-Actor: Actor-Centric Portrait Animation

• X-Actor is an audio-driven portrait animation framework that produces long-form, emotionally expressive talking head videos from a single reference image and audio input.
• It decouples the process into autoregressive motion prediction and a diffusion-based video synthesis module to maintain facial identity and extended emotional dynamics.
• The framework addresses long-range temporal coherence using diffusion forcing and specialized attention mechanisms, achieving competitive metrics in lip-sync and visual quality.

X-Actor most specifically denotes an audio-driven portrait animation framework that generates lifelike, emotionally expressive talking head videos from a single reference image and an input audio clip (Zhang et al., 4 Aug 2025). In that usage, the system is designed not merely for short-range lip synchronization, but for long-form portrait performance with nuanced, dynamically evolving emotions that remain coherent with the rhythm and content of speech. At the same time, the wider literature uses closely related “actor-centric” or “actor-based” terminology for a broader family of methods in reinforcement learning, video understanding, humanoid control, and distributed systems. This suggests that X-Actor is both a specific model name and a recurring design orientation in which the actor is treated as the primary unit of generation, reasoning, control, or coordination (Plisnier et al., 2019).

1. Definition and problem setting

In its named formulation, X-Actor addresses audio-driven portrait acting rather than generic talking-head synthesis. The stated objective is to generate long-form, emotionally expressive, actor-quality facial performances from a single reference image and an input audio clip, while avoiding the common failure modes of prior methods that emphasize lip synchronization and short-range visual fidelity in constrained speaking scenarios (Zhang et al., 4 Aug 2025). The target behavior includes nuanced expression, head motion, non-mouth facial dynamics, and temporally extended emotional transitions.

The central decomposition is explicit. X-Actor separates the problem into two stages: first, it predicts a temporally coherent sequence of facial motion latents driven by audio; second, it renders those motions into photorealistic video with a diffusion video synthesizer. The paper argues that this decoupling is what allows long-range emotion and acting dynamics to be modeled without entangling them with identity and appearance synthesis (Zhang et al., 4 Aug 2025).

A common source of confusion is that the term does not denote a single cross-domain formalism. In the supplied literature, “X-Actor” also functions as a loose descriptor for actor-centered reasoning modules and actor-based abstractions, but those uses refer to different technical objects: policy-gradient actors shaped by advice, actor-centric relation networks, actor-transformers, and actor runtimes are conceptually related by their actor-first orientation rather than by a shared architecture (Sun et al., 2018).

2. Core architecture

The first stage of X-Actor is an audio-conditioned autoregressive diffusion model for motion prediction. The motion sequence is factorized over chunks as

$$\phi(\mathbf{z}^{1:T} \mid \mathbf{a}^{1:T}) = \prod_{c=1}^{C} \phi(\mathbf{z}^{s_c:e_c} \mid \mathbf{z}^{<s_c}, \mathbf{a}^{<e_c}),$$

where the video is split into $C$ chunks and $\mathbf{z}^{s_c:e_c}$ is the current chunk of motion latents (Zhang et al., 4 Aug 2025). The model backbone is Qwen-2.5 7B adapted for temporal generation. It uses full self-attention inside each chunk, causal cross-chunk attention to prior generated chunks, and cross-attention to audio embeddings so that each motion token can attend to localized audio context.

Audio is encoded with Wav2Vec2 and projected into frame-aligned embeddings $\mathbf{a}^{1:T}$. For each motion token $z^t$, the model attends to a local window $\mathbf{a}^{t-2:t+2}$, which is intended to preserve precise lip sync while retaining broader temporal context (Zhang et al., 4 Aug 2025). The motion representation is deliberately compact and identity-agnostic. Following X-NeMo, a motion encoder maps multiscale facial motion into a latent vector $\mathbf{z} \in \mathbb{R}^{512}$. The paper argues that this latent is more expressive than landmarks or 3D coefficients while remaining more tractable than per-frame pixel latents for long-range autoregressive diffusion.

The second stage is a diffusion-based video synthesis module built on the Latent Diffusion Model framework. It uses an image autoencoder, a Stable Diffusion UNet, temporal attention layers following AnimateDiff, a ReferenceNet that extracts multi-scale features from the single input image $I_s$, and newly introduced cross-attention layers that consume the predicted motion latents $\mathbf{z}$ (Zhang et al., 4 Aug 2025). The functional separation is strict: the first stage determines what the face should do, and the second stage determines how that motion should be rendered while preserving identity and appearance.

3. Long-range temporal modeling and diffusion-forcing

The distinctive training problem in X-Actor is long-range autoregressive stability. The paper identifies error accumulation as the principal weakness of standard autoregressive training: during inference the model conditions on its own imperfect outputs, whereas during training it is exposed to clean history. X-Actor addresses this with diffusion forcing, which trains the model to predict future motion even when past context is noisy (Zhang et al., 4 Aug 2025).

For a single chunk, the motion predictor is trained with a diffusion objective over continuous latent chunks: $$\mathbb{E}_{\mathbf{z}^{s_c:e_c}, \epsilon_k, k} \left[ \left\|\,\mathbf{v}_k - v_\theta\left(\mathbf{z}_{k}^{s_c:e_c}, k, \mathbf{z}^{<s_c}, \mathbf{a}^{<e_c}\right)\,\right\|^2 \right].$$ The full diffusion-forcing objective corrupts the entire motion sequence with independently sampled noise levels across chunks: $C$0 Each chunk receives its own noise level $C$1, and all chunks are trained in parallel (Zhang et al., 4 Aug 2025). The intended effect is that the model does not overfit to perfect ground-truth history and instead learns to rely more heavily on audio cues.

Within each chunk, X-Actor uses asynchronous diffusion, meaning that different latents can receive different noise levels rather than one shared timestep. At inference time, motion generation uses DDIM, Classifier-Free Guidance, and a monotonically decreasing noise schedule over historical context so that distant history is noisier and recent history is cleaner (Zhang et al., 4 Aug 2025). The paper presents this as a mechanism for maintaining global emotional continuity while preserving local temporal smoothness.

This long-range design is central to the model’s claim of “portrait acting.” The paper attributes expressive performance to the interaction of three factors: a rich latent motion space, long-range autoregressive context, and audio-conditioned diffusion with CFG and diffusion forcing. A plausible implication is that X-Actor treats emotional motion not as a frame-local synchronization problem but as a long-horizon latent trajectory estimation problem.

4. Data, evaluation, and empirical profile

The training corpus is curated from HDTF, CelebV-HQ, and additional online sources, with SyncNet used to remove poor audio-visual alignment (Zhang et al., 4 Aug 2025). The final dataset contains 160K short clips under 10 seconds, averaging 5 seconds and totaling 260 hours, together with 280K long-form clips from 10 seconds to 1 minute, averaging 15 seconds and totaling 900 hours.

Evaluation is divided into self-driven and cross-driven settings. The self-driven setting is based on RAVDESS, using 100 identity-consistent audio-video pairs in which a frame from an actor is paired with a different audio clip from the same actor. The cross-driven in-the-wild setting uses 50 portrait images with varied identities, styles, and poses, together with 50 emotionally rich multilingual audio clips from movies, television, and songs, with durations from 20 to 200 seconds (Zhang et al., 4 Aug 2025). Baselines include SadTalker, JoyVasa, EchoMimic, Hallo3, MEMO, and Sonic.

The reported metrics include SynC and SynD for lip-sync quality, FVD for video quality, Glo and Exp for global motion and dynamic expression, DGlo and DExp for self-driven motion statistics difference, DEmo for emotion alignment via EMONet, and user-study metrics Syn, Div, Emo, and VQ (Zhang et al., 4 Aug 2025). On the self-driven benchmark, X-Actor achieves SynC $C$2, SynD $C$3, DGlo $C$4, and DEmo $C$5, each reported as best among the compared methods, while Sonic has a lower FVD value, $C$6 versus X-Actor’s $C$7. On the cross-driven benchmark, X-Actor reports SynC $C$8, Glo $C$9, Exp $\mathbf{z}^{s_c:e_c}$0, and user-study scores Syn $\mathbf{z}^{s_c:e_c}$1, Div $\mathbf{z}^{s_c:e_c}$2, Emo $\mathbf{z}^{s_c:e_c}$3, and VQ $\mathbf{z}^{s_c:e_c}$4 (Zhang et al., 4 Aug 2025).

The ablation study is organized around the model’s main claims. Replacing full intra-chunk attention with fully causal attention reduces SynC to $\mathbf{z}^{s_c:e_c}$5 and worsens SynD to $\mathbf{z}^{s_c:e_c}$6. Replacing diffusion forcing with teacher forcing yields SynC $\mathbf{z}^{s_c:e_c}$7 and SynD $\mathbf{z}^{s_c:e_c}$8. Removing CFG reduces performance to SynC $\mathbf{z}^{s_c:e_c}$9 and SynD $\mathbf{a}^{1:T}$0. Synchronous inter-chunk diffusion remains close to the full model but is slightly weaker, and alternative history-guidance schedulers are also inferior (Zhang et al., 4 Aug 2025). These results support the paper’s argument that long-range stability is an architectural rather than merely a rendering problem.

5. Interpretation, limitations, and common misconceptions

The most important conceptual distinction in X-Actor is between motion modeling and image synthesis. The system is not presented as an end-to-end video generator that directly maps audio and a portrait to frames. Instead, it treats expressive facial motion as an intermediate latent process, then renders that process with a separate diffusion synthesizer (Zhang et al., 4 Aug 2025). This design makes the model closer to a staged generative control system than to a conventional lip-sync module.

A common misconception is to treat X-Actor as only a higher-fidelity talking-head model. The supplied description is narrower and stronger: the framework is intended for “actor-quality, long-form portrait performance” and explicitly targets dynamically evolving emotion rather than short-range mouth accuracy alone (Zhang et al., 4 Aug 2025). Another misconception is that emotional expressiveness is attributed solely to the rendering backbone. The paper instead assigns a central role to the motion latent space, autoregressive diffusion, diffusion forcing, and time-adaptive guidance.

The limitations are explicit. X-Actor is currently limited to talking heads and does not model full-body motion or gestures. Expressiveness remains constrained by the underlying video diffusion backbone. The system does not yet fully capture extreme affective behaviors such as crying or trembling, does not generalize to non-human subjects like animals, and remains dependent on the availability of high-quality long-form emotional acting data (Zhang et al., 4 Aug 2025). These caveats are significant because they delimit the scope of the reported gains: the method advances portrait acting, not general embodied performance.

6. Related uses of the actor-centric idea

The broader literature shows that X-Actor belongs to a wider actor-centric family of methods, though those methods solve very different problems. In reinforcement learning, the Actor-Advisor architecture defines a policy-gradient actor trained from unbiased Monte-Carlo returns whose executed policy is a mixture of a learned policy $\mathbf{a}^{1:T}$1 and external advice $\mathbf{a}^{1:T}$2, with the advisory policy often derived from an off-policy Double DQN critic via Softmax policy shaping (Plisnier et al., 2019). In that setting, the actor remains policy-gradient based, but its trajectory is shaped by off-policy advice.

In video understanding, the actor serves as the anchor for relation reasoning. Actor-Centric Relation Network computes pairwise interactions between an actor proposal and all spatial locations in the feature map, then aggregates them with convolutions to improve spatio-temporal action localization (Sun et al., 2018). Actor-Context-Actor Relation Network extends this to higher-order reasoning by modeling the relation between two actors through shared context locations, while actor-transformers use self-attention over actor-level pose and I3D features to recognize individual actions and group activities (Pan et al., 2020). Actor-identified Spatiotemporal Action Detection further requires persistent actor identity, combining multiple object tracking with action classification so that the output specifies not only what action occurs where and when, but also who performs it (Yang et al., 2022).

Other papers generalize the actor-centric principle beyond visual analysis. ExoActor treats exocentric video generation as a high-level interface for humanoid control: instruction and scene observation are converted into imagined third-person video, then into estimated human motion, and finally into executable behavior via SONIC (Zhou et al., 30 Apr 2026). In HOI detection, an actor-centric framework conditions full-image reasoning on a selected human through an RGBM input and combines actor and object verb maps by center-point indexing (Xu et al., 2022). In distributed and concurrent systems, actor terminology shifts from perception to programming abstractions: task and actor programming models are described as duals in which the main difference is where synchronization logic resides (Yadav et al., 22 Aug 2025); OpenCL actors wrap OpenCL kernels in CAF so that GPUs become first-class execution targets within the actor model (Hiesgen et al., 2017); behavioral types and actor capabilities constrain message handling and message ordering in actor systems [(Crafa, 2012); (Gordon, 11 Feb 2025)].

Taken together, these usages show a stable underlying motif. Whether the domain is portrait animation, reinforcement learning, action localization, humanoid control, HOI detection, or distributed runtimes, the actor is treated as the privileged unit around which motion, context, interaction, or synchronization is organized. That continuity does not make these methods interchangeable, but it does explain why the name X-Actor is repeatedly attached to systems that prioritize actor-centered structure over purely scene-centric or globally symmetric formulations.