Perm: A Parametric Representation for Multi-Style 3D Hair Modeling (2407.19451v6)

Abstract: We present Perm, a learned parametric representation of human 3D hair designed to facilitate various hair-related applications. Unlike previous work that jointly models the global hair structure and local curl patterns, we propose to disentangle them using a PCA-based strand representation in the frequency domain, thereby allowing more precise editing and output control. Specifically, we leverage our strand representation to fit and decompose hair geometry textures into low- to high-frequency hair structures, termed guide textures and residual textures, respectively. These decomposed textures are later parameterized with different generative models, emulating common stages in the hair grooming process. We conduct extensive experiments to validate the architecture design of Perm, and finally deploy the trained model as a generic prior to solve task-agnostic problems, further showcasing its flexibility and superiority in tasks such as single-view hair reconstruction, hairstyle editing, and hair-conditioned image generation. More details can be found on our project page: https://cs.yale.edu/homes/che/projects/perm/.

• The paper introduces Perm, a new parametric model that disentangles global hair shape and local strand details using frequency-domain PCA.
• It integrates StyleGAN2 for guide strand synthesis and a VAE for modeling high-frequency textures, significantly improving reconstruction fidelity.
• The model enables precise hair editing, seamless hairstyle interpolation, and realistic single-view hair reconstruction for diverse applications.

Overview of Perm: A Parametric Representation for Multi-Style 3D Hair Modeling

The paper introduces a novel approach named Perm, a parametric representation that innovatively models 3D human hair. This is accomplished by leveraging a disentangled parameterization scheme using Principal Component Analysis (PCA) in the frequency domain to control both global hair shape and local strand details. The Perm model facilitates hair-related applications, such as 3D hair parameterization, hairstyle interpolation, and single-view hair reconstruction.

Perm distinguishes itself from previous methods by proposing a multi-level approach that clearly separates global and local hair features. This separation allows for precise control and editing capabilities of 3D hairstyles, addressing a notable limitation in current hair modeling methods. The paper details the architecture and training of Perm, including the use of StyleGAN2 for guide strand synthesis and a Variational Autoencoder (VAE) to model the complex local hair textures that encode high-frequency details.

Technical Contributions

• Disentangled Hair Representation: Unlike methods that treat global shapes and local details jointly, Perm provides a hierarchical representation using frequency-domain PCA. This offers an intuitive explanation for hairstyles, capturing both low-frequency macroscopic shapes and high-frequency details such as curliness.
• Guide Strand and Residual Texture Generation: The proposed model features separate networks to handle different modeling stages. A StyleGAN2 network is used for guide strand creation, and a VAE for high-resolution details, effectively mimicking complex artistic processes used in hair modeling software.
• Applications of Perm: The authors showcase the flexibility of Perm by deploying it across various applications. The model was not only able to parameterize 3D hair models accurately but also interpolated between styles seamlessly. Furthermore, Perm demonstrated utility in single-view reconstructions and hair-conditioned image generation—leveraging its generated parameters as priors in novel tasks.

Evaluation

The paper presents rigorous evaluation metrics and comparisons with existing methods. The guide strands generated using StyleGAN2 yield significantly lower position and curvature errors compared to PCA-based techniques. Similarly, the VAE outperforms StyleGAN2 when modeling complex residual textures, showcasing Perm's adaptability to different representation challenges.

Numerically, the model shows substantial improvements, with better reconstruction fidelity compared to comparable models like GroomGen. The results are vividly exemplified with interpolations showing smooth transitions between varied geometric styles, and single-view reconstructions that produce realistic hair models closely akin to reference images.

Implications and Future Work

The Perm model introduces a new paradigm in 3D hair modeling, offering enhanced control and flexibility. Its disentangled design philosophy may inspire similar applications in other digital human modeling domains. Future research directions could explore expanding the Perm model to incorporate complex hairstyles like buns and braids that encode additional structural intricacies not yet accounted for in the training dataset.

Perm's detailed representation also suggests potential applications in realistic virtual hair synthesis and gaming, where bespoke modeling fidelity is crucial. Further exploration into multi-modal control, integrating modalities such as semantic attributes or natural language, could pave the way for controllable 3D hair synthesis, enhancing user interactivity and customization capabilities.

In conclusion, Perm stands out as a statistically grounded and technologically robust framework that meaningfully advances the state of 3D hair modeling. Its ability to model a broad spectrum of hair styles while maintaining precision elucidates significant contributions to the field of computational graphics and digital hair modeling.