A Style-Based Generator Architecture for Generative Adversarial Networks (1812.04948v3)

Abstract: We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the generated images (e.g., freckles, hair), and it enables intuitive, scale-specific control of the synthesis. The new generator improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation. To quantify interpolation quality and disentanglement, we propose two new, automated methods that are applicable to any generator architecture. Finally, we introduce a new, highly varied and high-quality dataset of human faces.

• The paper proposes a style-based generator architecture that replaces traditional latent inputs with a learned constant and an intermediate mapping network using AdaIN.
• The paper demonstrates a 34% improvement in FID on CelebA-HQ and achieves robust disentanglement through mixing regularization and per-layer noise inputs.
• The approach offers enhanced image quality, flexible style control, and valuable applications in style transfer and unsupervised attribute separation.

A Style-Based Generator Architecture for Generative Adversarial Networks

The paper proposes a new generator architecture for generative adversarial networks (GANs) that incorporates concepts from style transfer literature. The architecture, which eschews traditional latent code input structures, demonstrates significant improvements in generative quality and disentanglement of latent factors. This essay explores the critical aspects, empirical findings, and implications of this proposed architecture.

Architectural Innovations

The proposed generator architecture departs from traditional designs by entirely omitting the input layer, instead initiating the synthesis from a learned constant. The design incorporates a non-linear mapping network that transforms the input latent space $Z$ into an intermediate latent space $W$. Each layer in the synthesis network is controlled by styles derived from $W$ using adaptive instance normalization (AdaIN). Additionally, stochastic variation is introduced through explicit noise inputs at each layer, enhancing the generator's capacity to produce fine-grained details.

Performance and Metrics

The efficacy of the style-based generator was evaluated against several benchmarks, primarily using the Fréchet Inception Distance (FID). The findings are as follows:

• Baseline Comparison: The improved baseline (configuration #1{b}) using bilinear up/down-sampling and extended training yielded a 34% improvement in FID on CelebA-HQ.
• Addition of Mapping Network and AdaIN: Incorporating these led to further improvements, with configurations emulating progressive architectural simplification (configuration #1{c} and #1{d}).
• Noise Inputs: Introducing per-channel noise inputs (configuration #1{e}) noticeably refined the generator's ability to render fine details.
• Mixing Regularization: The final configuration (#1{f}) utilizing style mixing regularization achieved the most significant improvements, indicating robust disentanglement and feature localization.

Theoretical Contributions

The theoretical advancements are underscored by the introduction of novel metrics to quantify merging latents' quality (perceptual path length) and latent space's linear separability. These metrics provide insights into the interplay between latent space transformations and image synthesis.

Perceptual Path Length

This metric computes the perceptual changes along interpolated latent paths, suggesting that a more disentangled $W$ space will show smoother transitions:

$l_{W} = \EX\Big[{\frac{1}{\epsilon^2}d\big(g(lerp(f(z_1), f(z_2); t)), g(lerp(f(z_1), f(z_2); t + \epsilon))\big)\Big]$

Empirical results show substantial reductions in perceptual path length with the proposed generator, affirming the linearity and reduced entanglement in $W$.

Linear Separability

The linear separability metric evaluates the ease with which distinct factors of variation can be isolated using a binary classifier. This is crucial for validating the disentanglement properties of $W$. The separability scores confirm that the style-based generator attains higher separability across latent factors.

Practical Implications

The practical implications of this work are considerable:

• Image Quality: The architecture demonstrates superior image quality and diversity, validated across datasets such as CelebA-HQ and the newly introduced FFHQ dataset.
• Mixing and Interpolation: The ability to mix styles at specific layers bolsters the generator's flexibility, allowing for refined control over the synthesis process, enhancing applications needing precise image manipulation.
• Disentanglement: The explicit disentanglement in $W$ suggests use cases for style transfer and unsupervised attributes separation, enriching tasks requiring fine attribute manipulations.

Future Directions

The findings open several avenues for future research:

1. Regularization Techniques: Incorporating the perceptual path length as a regularizer to directly shape the intermediate latent space.
2. Enhanced Architectures: Further exploration of different mappings and disentangling techniques could yield even better image synthesis and understanding.
3. Broader Applications: Application of this architecture to other data domains beyond faces, such as generative models for text and audio.

Conclusion

This paper showcases substantial improvements over traditional GAN architectures through an innovative generator design. By embedding latent codes in an intermediate latent space and employing AdaIN and noise inputs, the proposed model offers enhanced image quality, better interpolation, and significant disentanglement. These advancements not only push the boundaries of what is possible in image synthesis but also provide a robust framework for further exploration in generative models.

The complete findings, methodologies, and experimental results encapsulated in this paper significantly contribute to ongoing research in the field of GANs, setting a new benchmark for future endeavors in generative modeling.