One-shot Compositional 3D Head Avatars with Deformable Hair

Abstract: We propose a compositional method for constructing a complete 3D head avatar from a single image. Prior one-shot holistic approaches frequently fail to produce realistic hair dynamics during animation, largely due to inadequate decoupling of hair from the facial region, resulting in entangled geometry and unnatural deformations. Our method explicitly decouples hair from the face, modeling these components using distinct deformation paradigms while integrating them into a unified rendering pipeline. Furthermore, by leveraging image-to-3D lifting techniques, we preserve fine-grained textures from the input image to the greatest extent possible, effectively mitigating the common issue of high-frequency information loss in generalized models. Specifically, given a frontal portrait image, we first perform hair removal to obtain a bald image. Both the original image and the bald image are then lifted to dense, detail-rich 3D Gaussian Splatting (3DGS) representations. For the bald 3DGS, we rig it to a FLAME mesh via non-rigid registration with a prior model, enabling natural deformation that follows the mesh triangles during animation. For the hair component, we employ semantic label supervision combined with a boundary-aware reassignment strategy to extract a clean and isolated set of hair Gaussians. To control hair deformation, we introduce a cage structure that supports Position-Based Dynamics (PBD) simulation, allowing realistic and physically plausible transformations of the hair Gaussian primitives under head motion, gravity, and inertial effects. Striking qualitative results, including dynamic animations under diverse head motions, gravity effects, and expressions, showcase substantially more realistic hair behavior alongside faithfully preserved facial details, outperforming state-of-the-art one-shot methods in perceptual realism.

• The paper presents explicit face-hair decoupling to reduce animation artifacts and ensure clear separation between facial and hair components.
• It employs dual image-to-3D lifting with FLAME mesh registration and 3D Gaussian Splatting to achieve high-fidelity reconstructions.
• Experimental results show state-of-the-art performance with superior PSNR, SSIM, and realistic gravity-driven hair dynamics.

One-shot Compositional 3D Head Avatars with Deformable Hair: An Authoritative Technical Overview

Introduction and Motivation

This paper introduces a compositional strategy for constructing high-fidelity animatable 3D head avatars from a single frontal image, targeting deficits in previous one-shot approaches—primarily the failure to decouple hair from the facial region, which leads to entangled geometry, poor hair dynamics, and unrealistic appearance artifacts during animation. The method leverages explicit separation of hair and bald face, reconstructs each via specialized deformation paradigms, and seamlessly integrates them into a unified differentiable rendering and animation pipeline.

The key innovation is the explicit modeling of hair and face components as distinct primitives. The face is expressed with FLAME mesh-driven non-rigid registration, supporting natural deformation, while hair Gaussians are isolated and driven by physics-based Position-Based Dynamics (PBD), enabling realistic, physically plausible hair motion under head pose, gravity, and inertia.

Figure 1: Method overview—explicit decoupling and separate reconstruction of hair and bald face components using 3D Gaussian Splatting, FLAME mesh registration, and cage-based PBD for hair.

Method: Component Separation, Reconstruction, and Assembly

Dual Image-to-3D Lifting

A 2D bald filter generates a hairless variant of the input. Both the original and bald images are reconstructed via FaceLift into dense, detail-rich 3D Gaussian Splatting (3DGS) representations, preserving fine-grained textures and geometry. The bald part is then non-rigidly registered to a FLAME mesh with guidance from a prior model trained on large-scale datasets, enabling mesh-local Gaussian definition and natural expression-driven deformation.

Hair Isolation and Boundary Cleanup

Hair Gaussians are disentangled using semantic label supervision from multi-view images, augmented by a boundary-aware reassignment strategy to eliminate residual skin contamination caused by inaccurate segmentation. Specifically, 3D boundary regions are extracted via 2D boundary and depth cues, and boundary Gaussians are reclassified based on proximity to hair or skin features.

Figure 2: Hair cleanup using boundary-aware reassignment eliminates skin contamination and enables smooth hair-face transitions.

Component Alignment and Optimization

The dual reconstruction pipeline introduces shape inconsistencies between the bald and original images. Rough alignment is achieved via transformation of FLAME coefficients, with further joint optimization of hair Gaussians and FLAME parameters to ensure geometric consistency and prevent scalp-hair interpenetration. Collision penalties enforce externality of hair relative to the head mesh.

Cage-Based Hair Deformation with PBD

For long hair, rigid transformations are insufficient. A cage structure is constructed enclosing the hair Gaussians, with cage vertices treated as particles subject to PBD simulation. Root vertices (near the scalp) are kinematic and directly skin to the FLAME mesh via Linear Blend Skinning (LBS); free vertices undergo semi-implicit Euler integration and iterative constraint projection (stretch, bending, collision), yielding physically plausible hair dynamics.

A proxy-based collision constraint is introduced: Instead of enforcing collisions at cage-vertex positions—which would induce unrealistic gaps due to cage-Gaussian spatial misalignment—MVC interpolation computes proxy points representing Gaussian centers, and collision constraints are applied at these proxies, sharply improving contact realism.

Figure 3: Proxy-based collision constraint—collision detection and correction performed at interpolated proxy points rather than cage vertices, maintaining realistic skin-hair proximity.

Experimental Results and Empirical Analysis

Quantitative Metrics

On monocular video datasets (NeRSemble, Ava256, VFHQ), the method attains superior scores compared to prominent baselines including GPAvatar, Portrait4D-v2, GAGAvatar, CAP4D, and LAM:

• PSNR: 20.79
• SSIM: 0.864
• LPIPS: 0.124
• CSIM: 0.824
• AED, APD, AKD: State-of-the-art performance for fidelity in expression and pose transfer

Qualitative results demonstrate preservation of high-frequency facial and hair details, outstanding volume and side-view quality, and physically plausible gravity-driven hair motion absent in prior holistic methods.

Figure 4: Cross-reenactment comparison illustrating improved fidelity and realistic hair dynamics.

Figure 5: Qualitative self-reenactment comparison—PBD-based deformation yields natural hair sway, avoiding rigid deformation artifacts found in other methods.

Ablation Studies

• Hair Deformation (Figure 6): Static hair or naive collision constraints lead to nonphysical rigidity and interpenetration. Proxy-based collision constraint achieves stable, penetration-free, and realistic hair dynamics.
• Geometric Consistency Loss $\mathcal{L}_{geo}$ (Figure 7): Joint optimization prevents hair Gaussians from penetrating the occipital region, enforcing smooth geometry.
• Boundary-aware Reassignment (Figure 8): Removes residual skin artifacts in hair Gaussians, improving cross-identity hairstyle transfers and boundary smoothness.

  Figure 6: Ablation on hair deformation—proxy-based collision constraint successfully avoids nonphysical gaps and achieves realistic dynamics.

  

  Figure 7: Ablation on $\mathcal{L}_{geo}$—optimizing hair together with FLAME prevents occipital interpenetration.

  

  Figure 8: Boundary-aware reassignment removes skin artifacts, yielding smooth, natural hair transitions and improved cross-identity transfer.

Implications and Future Directions

This compositional approach enables component-wise control and animation from single images, supports real-time rendering from arbitrary viewpoints and expressions, and achieves physically plausible hair simulation without data-driven training. It closes critical gaps in one-shot avatar construction:

• Enables explicit face-hair decoupling, preventing deformation artifacts
• Introduces simulation-driven cage-based deformation for volumetric hair, integrated with differentiable rendering
• Offers superior cross-identity transferability and preserves person-specific fidelity

Future theoretical directions include advancing strand-level hair dynamics, physical simulation of interactions, and generalization to unconstrained or occluded inputs. Practically, this model facilitates immersive avatars for virtual try-on, gaming, VR, and telepresence with refined realism and dynamics.

Conclusion

This paper establishes a compositional, physics-driven framework for one-shot animatable 3D head avatars, solving persistent challenges in holistic modeling and hair dynamics. Its dual lifting, boundary-aware segmentation, and PBD cage simulation robustly unify facial and hair animation under coherent rendering, setting a new benchmark for single-image avatar reconstruction (2604.14782).