ToonAging: Face Re-Aging upon Artistic Portrait Style Transfer (2402.02733v4)

Abstract: Face re-aging is a prominent field in computer vision and graphics, with significant applications in photorealistic domains such as movies, advertising, and live streaming. Recently, the need to apply face re-aging to non-photorealistic images, like comics, illustrations, and animations, has emerged as an extension in various entertainment sectors. However, the lack of a network that can seamlessly edit the apparent age in NPR images has limited these tasks to a naive, sequential approach. This often results in unpleasant artifacts and a loss of facial attributes due to domain discrepancies. In this paper, we introduce a novel one-stage method for face re-aging combined with portrait style transfer, executed in a single generative step. We leverage existing face re-aging and style transfer networks, both trained within the same PR domain. Our method uniquely fuses distinct latent vectors, each responsible for managing aging-related attributes and NPR appearance. By adopting an exemplar-based approach, our method offers greater flexibility compared to domain-level fine-tuning approaches, which typically require separate training or fine-tuning for each domain. This effectively addresses the limitation of requiring paired datasets for re-aging and domain-level, data-driven approaches for stylization. Our experiments show that our model can effortlessly generate re-aged images while simultaneously transferring the style of examples, maintaining both natural appearance and controllability.

• The paper introduces a unified, one-stage GAN framework that combines face re-aging with artistic style transfer via a novel latent fusion mechanism.
• It employs an exemplar-based approach using the W+ latent space to preserve facial attributes and seamlessly infuse non-photorealistic styles.
• Experimental results demonstrate that ToonAging outperforms sequential methods, offering practical advantages for animation and creative industries.

Insights into ToonAging: A Unified Approach to Artistic Portrait Style Transfer and Face Re-Aging

ToonAging is a proposed framework unifying the tasks of face re-aging and non-photorealistic rendering (NPR) style transfer within a single generative approach. This domain presents challenges due to the complexities of re-aging coupled with artistic stylization, necessitating sophisticated methods to preserve image quality and attributes.

Core Contributions and Methodology

This paper introduces a one-stage method that innovatively combines face re-aging with portrait style transfer. Unlike traditional methods that rely on sequential operations—first altering the age and then transferring the artistic style, or vice versa—ToonAging integrates these processes into a single generative step. This integration leverages distinct latent spaces in a generative adversarial network (GAN) paradigms, specifically the StyleGAN-based architectures. Here's a closer look at the key contributions:

• Latent Fusion Mechanism: By adopting a latent fusion process, ToonAging efficiently combines re-aging with stylization across diverse domains without requiring separate networks for each stylization goal. The model simultaneously manipulates latent vectors responsible for aging and NPR characteristics to ensure coherent output.
• Exemplar-based Approach: The method stands out by employing an exemplar-based technique, allowing it to assimilate styles from various references without extensive fine-tuning or additional datasets. This aspect is crucial in contexts with limited access to large training datasets and supports broad adaptability across different art styles.
• Use of W+ Latent Space: The approach utilizes the W+ latent space of StyleGAN more effectively than the Z+ space, achieving better preservation of facial attributes, ensuring the fidelity of re-aging while maintaining stylistic integrity.

Experimental Validation

Experiments conducted highlight the model's prowess in generating aesthetically pleasing and realistic re-aged images without compromising the essence of the input style. User studies verified the ability of ToonAging to outperform naive sequential processing of re-aging followed by style transfer, or the reverse. The results indicate a significant improvement over other techniques in creating convincing NPR images with infused age characteristics.

Practical Implications and Future Directions

The implications of ToonAging are multi-faceted. Practically, this approach empowers creators in the entertainment industry to produce animated content more efficiently, enhancing artistic flexibility while minimalizing the manual workload typically associated with artistic re-aging of faces.

Theoretically, ToonAging opens avenues for further research into latent space manipulations and the interactions of multifaceted generative tasks within singular models. Future investigations could probe deeper into optimizing latent space encodings to enhance the stability and performance of generative models under various constraints.

Conclusion

ToonAging marks a significant stride in the intersection of computer vision and artistic rendering, comfortably handling dual objectives of re-aging and art style transformation. Its integration of exemplar-based learning within a single-stage process suggests a paradigm shift towards more efficient image synthesis frameworks. This paper sets the stage for subsequent explorations into GAN-based artistic transformations and style transfers, offering valuable insights into the practical application of AI techniques in creative industries.