DynamicFace: Dual 3D Avatar & Face Swapping

• DynamicFace is a dual-natured contribution addressing both monocular 3D avatar reconstruction and diffusion-based face swapping.
• In the GeoAvatar approach, it provides a high-resolution facial motion dataset emphasizing expressive mouth dynamics and varied head poses.
• The diffusion-based framework transfers identity while preserving expression, pose, and scene details to ensure consistent video synthesis.

Searching arXiv for the cited DynamicFace-related papers to ground the article in current arXiv records. DynamicFace is a name used in arXiv literature for two technically distinct 2025 contributions. In "GeoAvatar: Adaptive Geometrical Gaussian Splatting for 3D Head Avatar," DynamicFace is a monocular, high-resolution facial motion dataset introduced to stress-test and train 3D head avatar methods under highly expressive, mouth-centric dynamics and varied head poses (Moon et al., 24 Jul 2025). In "DynamicFace: High-Quality and Consistent Video Face Swapping using Composable 3D Facial Priors," DynamicFace is a diffusion-based image/video face swapping framework that transfers the identity of a source face to a target while preserving the target’s expression, head pose, hair, illumination, and background (Wang et al., 15 Jan 2025). The shared name therefore spans two adjacent but distinct subfields: monocular avatar reconstruction and animation, and identity transfer for generative face synthesis.

1. Nomenclature and scope

The two uses of DynamicFace differ in data modality, objective, and evaluation protocol, but both are organized around the relation between identity-preserving structure and dynamic facial variation.

Usage	Domain	Core function
DynamicFace in GeoAvatar	Monocular facial motion dataset	Stress-test and train 3D head avatar methods under highly expressive, mouth-centric dynamics and varied head poses
"DynamicFace: High-Quality and Consistent Video Face Swapping using Composable 3D Facial Priors"	Diffusion-based face swapping method	Transfer source identity while preserving target expression, pose, hair, illumination, and background

This dual usage is not merely terminological. The dataset version is designed to expose failure modes in rigged Gaussian avatar models, especially around mouth animation and viewpoint generalization, whereas the method version addresses target identity leakage and poor expression or pose fidelity in face swapping. A plausible implication is that the common name reflects a shared emphasis on dynamics, but the technical problems, architectures, and benchmarks are different.

2. DynamicFace as a monocular facial motion dataset

As introduced in GeoAvatar, DynamicFace is a monocular, high-resolution facial motion dataset collected to fill a gap in existing single-camera facial datasets that often lack sustained, diverse mouth motions and broad head pose changes needed to evaluate robust animation, not just reconstruction (Moon et al., 24 Jul 2025). The paper explicitly targets the hardest region—the mouth—where 3DMMs such as FLAME are anatomically incomplete and exhibit fitting errors. The dataset was collected to provide highly expressive, mouth-dynamic sequences for learning robust animation, to supply broad head-pose coverage with a single camera, and to benchmark animation fidelity, especially mouth fidelity, together with identity preservation under rapid or exaggerated expressions.

The dataset contains 10 actors, described as “10 videos, each recording a single actor.” Each subject is recorded for 2–3 minutes, for a total of approximately 32.25 minutes across all subjects. The videos are 3840×2160 RGB, with a total disk footprint of approximately 18.92 GB. Actors were instructed to perform a wide range of expressions and mouth actions and to slowly nod throughout the sequences. The resulting content includes wide open-mouth frames, strong jaw opening and closing, lip movements, visible teeth and mouth interior, subtle timing variations, and dynamic talking scenarios emphasized in the test splits. Eye and eyebrow movements are present, and accessory visibility, such as eyeglasses, appears in some subjects. Head pose variation is introduced by slow nodding to capture a range of yaw, pitch, and roll views with a single camera; the paper reports successful novel-view synthesis for extreme facial degrees, although precise angle ranges are not numerically reported.

The capture setup is markerless and monocular. Nine subjects were recorded with a single Sony AX700 camcorder against a chromakey background, and one subject was recorded with a single iPhone14 with a normal background. The chromakey captures provide a consistent, controlled background; explicit lighting specifications are not reported. Audio is not reported as a provided modality, and GeoAvatar uses video-only inputs. There are no official subject-independent splits. In GeoAvatar benchmarking, each subject uses the last 350 frames as test, with the preceding frames used for training.

A quantitative comparison reported alongside GeoAvatar positions DynamicFace against commonly used facial datasets:

Dataset	IDs / expressions	Resolution / duration / size
NerFace	3 IDs, 1 expression	1920×1080, 6.44 min, 3.79 GB
IMAvatar	4 IDs, 11 expressions	512×512, 7.08 min, 3.39 GB
DynamicFace	10 IDs, ≈20 expression categories	3840×2160, 32.25 min, 18.92 GB

Within the paper’s framing, these properties make DynamicFace a stricter benchmark for animation robustness and mouth fidelity than earlier monocular sets.

3. GeoAvatar integration: preprocessing, rigging, and evaluation

DynamicFace is distributed as videos, and the paper does not explicitly claim that precomputed labels such as FLAME parameters ship with the dataset (Moon et al., 24 Jul 2025). In the GeoAvatar pipeline, per-frame FLAME parameters are estimated from the videos via a modified MICA pipeline updated to 2023 FLAME, yielding shape $\beta$, expression $\psi$, pose $\Theta$ for jaw and eyes, global rotation $R$, translation $t$, and a tracked mesh with 5023 vertices and 9976 faces. Foreground masks are computed during preprocessing by intersecting Background Matting and BiSeNet outputs to produce sharp, human-only masks without background artifacts. Gaussians are then bound to FLAME faces and transformed from local to global space by face-level rotation, scale, and center derived from the tracked mesh.

GeoAvatar’s main methodological response to DynamicFace’s difficulty is Adaptive Pre-allocation Stage (APS), which separates rigid and flexible facial parts after a warm-up of $N=100k$ iterations. The part-wise mean offset magnitude is

$$\mathrm{Distance}(k)=\frac{1}{|\mathrm{p}_k|}\sum_{\mathbf{f}_i\in \mathrm{p}_k}\frac{1}{|G_i|}\sum_{j\in G_i}\left\|\boldsymbol\mu_{i,j}\right\|.$$

The mean of $\mathrm{Distance}(k)$ over parts defines the threshold $\tau_{\mathrm{part}}$; part $\mathrm{p}_k$ is assigned to the rigid set $\psi$0 if $\psi$1, and to the flexible set $\psi$2 otherwise. In the reported interpretation, well-fitted regions such as face and lips are rigid, while poorly fitted regions such as scalp, neck, and ears with long hair are flexible.

The paper also augments FLAME with a mouth structure containing frontal and molar teeth, palate, and mouth floor. This mouth structure set $\psi$3 is split into upper and lower parts, and the deformation rule is written as

$\psi$4

The offsets are predicted by deformation networks with positional encoding in time,

$\psi$5

which provides frame-specific correction beyond FLAME tracking accuracy while preserving intra-part anatomical coherence.

Rigging regularization is defined in spherical coordinates using the radius $\psi$6 and elevation angle $\psi$7 of local Gaussian means. The set-dependent radius constraint is

$\psi$8

and the angle constraint is

$\psi$9

The final regularizer is

$\Theta$0

and the training loss is

$\Theta$1

GeoAvatar evaluates DynamicFace through self-reenactment, cross-reenactment, and novel-view synthesis. Metrics include MSE, PSNR, SSIM, LPIPS, identity cosine similarity with InsightFace, and expression similarity with a FER2013-trained facial expression recognizer. On self-reenactment over 10 subjects with the last 350 frames as test, GeoAvatar reports MSE $\Theta$2, PSNR $\Theta$3, SSIM $\Theta$4, and LPIPS $\Theta$5, with mean and standard deviation reported in the paper. In cross-reenactment, the method obtains higher identity and expression similarity than the baselines. On NeRSemble, a 16-view dataset, GeoAvatar trained with just 1 view is reported as competitive with GaussianAvatars trained on all 16 views, with better SSIM and LPIPS and marginal differences in MSE and PSNR. The paper also reports real-time inference at more than 60 FPS.

4. DynamicFace as a diffusion-based face-swapping framework

The other DynamicFace is a diffusion-based image/video face swapping method designed to transfer the identity of a source face to a target face while retaining attributes such as expression, pose, hair, and background of the target (Wang et al., 15 Jan 2025). Its explicit targets are target identity leakage and poor expression or pose fidelity, particularly in video. The method is organized around four composable, disentangled 3D facial priors, a Stable Diffusion-based generation framework conditioned through a Mixture-of-Guiders and a fusion network, dual-stage identity injection via FaceFormer and ReferenceNet, and plug-and-play temporal attention from AnimateDiff for video consistency.

The pipeline begins with preprocessing and alignment. A source face image $\Theta$6 and target video frames $\Theta$7 are processed by 3DDFA-V3 to estimate identity $\Theta$8, pose $\Theta$9, expression $R$0, camera $R$1, and albedo or texture $R$2. Two-dimensional landmarks are used for expression segmentation, and BiSeNet is used for face parsing masks. From these reconstructions, the method constructs four disentangled conditions:

$R$3

The shape-aware normal map is

$R$4

which preserves source shape while aligning to target pose and camera, with a base or template expression to avoid expression leakage. The background condition uses an inpainting-style mask:

$R$5

$R$6

and

$R$7

The expression-related segmentation is

$R$8

where the design focuses on the inner mouth to avoid introducing identity-like lip thickness. The identity-erased illumination condition is

$R$9

which preserves lighting information while blurring identity-bearing UV detail.

The generative core is a Stable Diffusion latent UNet with VAE encoder and decoder operating at 512×512 resolution. Motion conditions are injected via lightweight condition encoders and a fusion net with self-attention; both end with zero-conv to preserve pretrained priors. Identity is injected at two granularities. FaceFormer uses an ArcFace-based face recognition encoder and a querying transformer with $t$0 learnable queries to produce high-level identity tokens for cross-attention. ReferenceNet is a trainable copy of the base UNet that processes the source image, with random irregular cutout augmentation, and injects dense appearance features through spatial-attention by replacing self-attention layers in the middle and upsample stages of the main UNet. For video, AnimateDiff temporal attention layers are inserted into the UNet and trained in a second stage to aggregate information across frames.

5. Training regime, empirical performance, and limitations

The diffusion formulation follows the latent diffusion objective

$t$1

with training loss

$t$2

Temporal attention is described generically as

$t$3

where the aggregation is used to improve identity and motion consistency across frames (Wang et al., 15 Jan 2025).

Training is two-stage. Stage 1 operates in the image domain on VGGFace2, VFHQ, and a private high-quality dataset, adapting Stable Diffusion to the face domain. It trains FaceFormer, ReferenceNet, the guiders, and the main UNet at 512×512 for 260,000 steps with batch size 32, Adam, and learning rate $t$4. Either the expression or normal condition is randomly dropped per sample to prevent dominance. Additional reconstruction losses are applied on the facial area and specific regions including eyes, ears, mouth, and nose. Stage 2 trains only the temporal attention layers on 16-frame clips for 40,000 steps with batch size 16 and the same learning rate. AnimateDiff temporal layers are initialized from pretrained weights. The reported hardware is 4× NVIDIA H800 GPUs. Typical identity features in FaceFormer use 112×112 crops.

Evaluation is performed on FaceForensics++ (FF++). Reported image-level metrics are ID retrieval, pose error, expression error, mouth position error, and eye motion error; video-level metrics are CLIP frame consistency and warping error. In the table reported by the paper, DynamicFace achieves ID 99.20, Pose 1.73, Expr. 3.08, Mouth 1.69, and Eye 0.16. The same table lists Deepfakes at ID 91.40, Pose 3.32, Expr. 5.02, Mouth 7.11, Eye 0.35; FaceShifter at ID 96.00, Pose 2.12, Expr. 3.49, Mouth 3.45, Eye 0.29; MegaFS at ID 81.62, Pose 5.33, Expr. 4.52, Mouth 9.34, Eye 0.31; SimSwap at ID 98.50, Pose 1.05, Expr. 2.85, Mouth 2.39, Eye 0.22; DiffSwap at ID 17.21, Pose 1.67, Expr. 3.05, Mouth 3.56, Eye 0.24; and Face Adapter at ID 98.69, Pose 1.90, Expr. 4.15, Mouth 4.08, Eye 0.23. The paper attributes the slightly higher pose error of DynamicFace to altered face shape affecting pose estimators.

The video ablation reports that removing the motion module yields ID retrieval 99.21, ID similarity 0.594, Pose 1.81, Expr. 3.12, Mouth 1.98, Eye 0.17, Frame consistency 95.78, and Warping error 0.091, whereas adding the motion module gives ID retrieval 98.90, ID similarity 0.574, Pose 1.54, Expr. 2.94, Mouth 2.30, Eye 0.16, Frame consistency 99.02, and Warping error 0.046. The identity injection ablation reports FaceFormer only at ID similarity 0.515, Pose 1.34, Expr. 3.18; ReferenceNet only at ID similarity 0.520, Pose 1.35, Expr. 3.23; and both modules at ID similarity 0.547, Pose 1.21, Expr. 2.32.

The paper also states several limitations. Reliance on 3D reconstruction accuracy can introduce mismatches if priors are imperfect. Extreme occlusions or lighting changes may challenge UV-based illumination and mask estimation. Disentanglement is achieved by design rather than explicit orthogonality losses, so minor leakage may remain. The ethics section emphasizes misuse risks in face swapping, including privacy and misinformation, and names consent, watermarking, and detection as safeguards.

6. Conceptual background: identity, expression, and dynamic facial representation

A relevant conceptual antecedent is the face-space duality hypothesis, which proposes that faces are encoded in one multidimensional space with an integral representation, yet this single space has a twofold structure that differentially supports dynamic expression information and invariant identity information (Vitale et al., 2016). In that model, a face image $t$5 is projected to

$t$6

and a permutation $t$7 reverses coordinate order so that leading dimensions emphasize expression-related variation while trailing dimensions emphasize identity-related variation. The formal duality conditions are stated as

$t$8

with $t$9.

The computational model is a supervised graph-based dimensionality reduction with PCA pre-reduction and a trace-ratio objective. Expression-similarity and identity-penalty graphs define

$N=100k$0

The learned basis minimizes

$N=100k$1

and the dual identity-favoring ordering is obtained from the same eigenvectors in reverse order. On KDEF, the reported component-usage analysis with $N=100k$2 and $N=100k$3 images showed components #1–#52 as predominantly used for expression, components #53–#99 as shared utility, and component #100 as predominantly used for identity.

This antecedent is not a direct precursor of either 2025 DynamicFace system, but it provides a useful interpretive frame. A plausible implication is that both the GeoAvatar dataset and the diffusion-based face swapping method instantiate, in different engineering forms, the same general problem: how to preserve invariant identity while modeling or transferring dynamic expression, pose, and motion. In the dataset setting, that problem appears as FLAME-guided Gaussian rigging and mouth-specific deformation under expressive motion. In the diffusion setting, it appears as disentangled 3D priors, dual identity injection, and temporal attention for consistent video synthesis.