cGANs with Projection Discriminator (1802.05637v2)

Abstract: We propose a novel, projection based way to incorporate the conditional information into the discriminator of GANs that respects the role of the conditional information in the underlining probabilistic model. This approach is in contrast with most frameworks of conditional GANs used in application today, which use the conditional information by concatenating the (embedded) conditional vector to the feature vectors. With this modification, we were able to significantly improve the quality of the class conditional image generation on ILSVRC2012 (ImageNet) 1000-class image dataset from the current state-of-the-art result, and we achieved this with a single pair of a discriminator and a generator. We were also able to extend the application to super-resolution and succeeded in producing highly discriminative super-resolution images. This new structure also enabled high quality category transformation based on parametric functional transformation of conditional batch normalization layers in the generator.

• The paper introduces a projection discriminator that embeds conditional information via inner products to outperform traditional concatenation methods.
• It achieves superior class conditional image generation and super-resolution with notable improvements in Inception Score and FID metrics.
• The approach aligns discriminator training with probabilistic models, offering a stable method and inspiring further advancements in GAN research.

Conditional GANs with Projection Discriminator: Enhanced Class Conditional Image Generation and Super-Resolution

The paper "cGANs with Projection Discriminator" by Takeru Miyato and Masanori Koyama presents an innovative approach to incorporating conditional information into the discriminator of Conditional Generative Adversarial Networks (cGANs). The primary contribution lies in using a projection-based method to leverage conditional information more effectively than traditional concatenation methods, leading to superior performance in class conditional image generation and super-resolution tasks.

Core Contributions

The authors propose a projection-based modification to the discriminator in cGANs. Traditionally, conditional information is concatenated with feature vectors either at the input layer or intermediate layers of the discriminator. This paper challenges that approach by embedding the conditional information as an inner product with the discriminator's feature vectors, aligning better with underlying probabilistic models. The primary contributions of this work can be summarized as follows:

1. Projection Discriminator: The introduction of a projection-based discriminator that respects the probabilistic nature of conditional variables.
2. Enhanced Image Generation: Demonstration of significant improvements in the quality of class conditional image generation on the ILSVRC2012 dataset.
3. High-Quality Super-Resolution: Application of the model to produce high-quality super-resolution images that are highly discriminative.
4. Category Morphing: Capability to perform high-quality category transformation using parametric functional transformation of conditional batch normalization layers.

Key Findings and Numerical Results

The authors compare their proposed method against conventional concatenation-based cGANs and Auxiliary Classifier GANs (AC-GANs). Key numerical results from their experiments include:

• Inception Score Improvement: For class conditional image generation on ImageNet, the proposed method achieved an Inception Score of 29.7, significantly outperforming concatenation-based approaches and AC-GANs.
• Intra-Class FID: The model demonstrated superior intra-class FID scores, indicative of better alignment with target distributions and reduced mode collapse.
• Super-Resolution Metrics: In the super-resolution task, the projection-based discriminator outperformed bicubic and concatenation methods in both inception accuracy (35.2%) and MS-SSIM scores (0.878).

Theoretical and Practical Implications

The authors emphasize that the projection-based method aligns the discriminator training with probabilistic assumptions about the data, resulting in more stable and effective training. Theoretical implications suggest that imposing such probabilistic regularization can lead to more realistic and diverse generative models.

Practically, this research highlights improvements in generating high-quality images for applications requiring detailed image generation, such as medical imaging or autonomous driving systems. The successful application to super-resolution also suggests potential in video enhancement and other multimedia applications.

Future Directions

The paper opens several avenues for future research:

1. Generalization to Other Tasks: Extending the approach to other applications such as semantic segmentation and image-to-image translation.
2. Enhanced Model Complexity: Increasing the complexity of the neural network models to further improve the visual quality and diversity of generated images.
3. Mode Collapse Mitigation: Continued investigation into disentangling mode collapse and improving distributional metrics for generative models.
4. Alternative Probabilistic Models: Exploring the impact of diverse probabilistic models on the discriminator's performance in various generative tasks.

Conclusion

The projection-based discriminator presented in this paper represents a notable advancement in conditional Generative Adversarial Networks. By adhering more closely to the underlying probabilistic models and leveraging inner-product embedding of conditional information, this approach shows substantial improvements over traditional methods in both image generation quality and super-resolution tasks. The implications of this work extend to numerous practical applications, and it sets the stage for continued exploration of discriminator architectures in generative modeling.