Stable-Hair: Real-World Hair Transfer via Diffusion Model (2407.14078v2)

Abstract: Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. In this paper, we propose a novel diffusion-based hair transfer framework, named \textit{Stable-Hair}, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on. To achieve this goal, our Stable-Hair framework is designed as a two-stage pipeline. In the first stage, we train a Bald Converter alongside stable diffusion to remove hair from the user-provided face images, resulting in bald images. In the second stage, we specifically designed a Hair Extractor and a Latent IdentityNet to transfer the target hairstyle with highly detailed and high-fidelity to the bald image. The Hair Extractor is trained to encode reference images with the desired hairstyles, while the Latent IdentityNet ensures consistency in identity and background. To minimize color deviations between source images and transfer results, we introduce a novel Latent ControlNet architecture, which functions as both the Bald Converter and Latent IdentityNet. After training on our curated triplet dataset, our method accurately transfers highly detailed and high-fidelity hairstyles to the source images. Extensive experiments demonstrate that our approach achieves state-of-the-art performance compared to existing hair transfer methods. Project page: \textcolor{red}{\url{https://xiaojiu-z.github.io/Stable-Hair.github.io/}}

• The paper presents a novel diffusion-based framework that achieves high fidelity in hair transfer compared to traditional GAN methods.
• It utilizes a two-stage pipeline with a Bald Converter and specialized Hair Transfer Modules to preserve intricate hairstyle details and identity consistency.
• Experimental results show superior performance in metrics such as FID, SSIM, and IDS, validating its real-world applicability and robustness.

Stable-Hair: Real-World Hair Transfer via Diffusion Model

This summary provides an insight into the academic paper titled "Stable-Hair: Real-World Hair Transfer via Diffusion Model." The research focuses on a novel methodology for robust and high-fidelity hair transfer in real-world scenarios. This is achieved using a diffusion-based framework that outperforms traditionally used GAN-based methods, addressing many of the shortcomings associated with handling intricate and diverse hairstyles.

Methodology

Stable-Hair introduces a two-stage pipeline designed to overcome the limitations inherent in previous methods:

1. Stage One: Bald Converter - Initially, the user-provided face image is transformed into a bald image using a Bald Converter guided by stable diffusion. This step is crucial for removing the existing hairstyle and preparing the image for the subsequent hair transfer process.
2. Stage Two: Hair Transfer Modules - This phase involves three bespoke modules:
   • Hair Extractor: This component is trained to encode the hairstyle from a reference image, ensuring that intricate and complex hairstyle details are preserved.
   • Latent IdentityNet: This module encodes the original face image to maintain the identity and background consistency between the source and the transformed image.
   • Hair Cross-Attention Layers: Integrated into the diffusion U-Net, these layers facilitate precise and high-fidelity transfer of the reference hairstyle to the bald image.

Moreover, the Latent ControlNet architecture replaces the traditional ControlNet to ensure that content preservation, especially color consistency in non-hair regions, is maintained throughout the two-stage process.

Experimental Validation

The efficiency and superiority of Stable-Hair are demonstrated through extensive experiments involving various data sets and comparisons with other state-of-the-art methods. Quantitative metrics, such as FID, PSNR, SSIM, and IDS, are employed to evaluate performance, with Stable-Hair showing overall improved results:

• FID (Fréchet Inception Distance): Stable-Hair achieved a score of 33.653, surpassing other methods like HairFastGAN (36.205) and HairclipV2 (37.456), indicating higher fidelity and realism in generated images.
• PSNR (Peak Signal-to-Noise Ratio): Scores were competitive, with Stable-Hair achieving 29.555, slightly lower than HairclipV2's 30.619 but superior in other measures.
• SSIM (Structural Similarity Index): Stable-Hair scored 0.640, emphasizing its ability to maintain structural integrity and identity.
• IDS (Identity Similarity): Achieved a score of 0.771, showing the system's ability to preserve identity content better than alternatives like SYH (0.712) and Hairclip (0.697).

Further qualitative comparisons and a comprehensive user paper confirmed the method’s robustness across various hairstyles and its utility in real-world applications. Stable-Hair consistently provided high-quality transfers by effectively preserving the structural and stylistic nuances of the reference hairstyles.

Contributions and Implications

The contributions of this paper are multifaceted:

• Diffusion-Based Hair Transfer: Introducing the first diffusion-based framework for hairstyle transfer, Stable-Hair merges the stable training capabilities of diffusion models with the specific requirements of hair transfer tasks.
• Latent ControlNet Architecture: By transitioning the task from pixel space to latent space, Stable-Hair ensures higher content consistency and eliminates color discrepancies, which are common pitfalls in previous methods.
• Automated Data Production Pipeline: A robust pipeline for generating training data ensures the system's effectiveness and adaptability to diverse real-world scenarios.

Future Directions

The paper opens several avenues for future developments in AI-centric image processing:

• Cross-Domain Applications: Future research could explore applying diffusion-based transfer techniques to domains beyond hairstyles, such as clothing or accessory transfer.
• Enhanced Training Data: Improving the training data to eliminate transfer of accessories and other non-hair features could enhance the fidelity and applicability of the transfer results.
• Ethical Considerations: Addressing privacy and consent concerns will be critical as such technologies become more pervasive.

Stable-Hair sets a new standard in virtual try-on experiences and personalized digital avatars by delivering precise, high-fidelity hairstyle transfers. Its innovative use of diffusion models signifies a significant step forward, potentially enabling broader applications and improved performance in various image synthesis and editing tasks.