Rebooting ACGAN: Auxiliary Classifier GANs with Stable Training (2111.01118v1)

Abstract: Conditional Generative Adversarial Networks (cGAN) generate realistic images by incorporating class information into GAN. While one of the most popular cGANs is an auxiliary classifier GAN with softmax cross-entropy loss (ACGAN), it is widely known that training ACGAN is challenging as the number of classes in the dataset increases. ACGAN also tends to generate easily classifiable samples with a lack of diversity. In this paper, we introduce two cures for ACGAN. First, we identify that gradient exploding in the classifier can cause an undesirable collapse in early training, and projecting input vectors onto a unit hypersphere can resolve the problem. Second, we propose the Data-to-Data Cross-Entropy loss (D2D-CE) to exploit relational information in the class-labeled dataset. On this foundation, we propose the Rebooted Auxiliary Classifier Generative Adversarial Network (ReACGAN). The experimental results show that ReACGAN achieves state-of-the-art generation results on CIFAR10, Tiny-ImageNet, CUB200, and ImageNet datasets. We also verify that ReACGAN benefits from differentiable augmentations and that D2D-CE harmonizes with StyleGAN2 architecture. Model weights and a software package that provides implementations of representative cGANs and all experiments in our paper are available at https://github.com/POSTECH-CVLab/PyTorch-StudioGAN.

• The paper identifies gradient explosion in ACGAN's classifier as a core challenge and introduces unit hypersphere feature normalization to stabilize training.
• It proposes a novel Data-to-Data Cross-Entropy (D2D-CE) loss that enhances intra-class variation and inter-class separability for robust learning.
• Empirical results on CIFAR10, Tiny-ImageNet, CUB200, and ImageNet demonstrate improved FID and Precision-Recall metrics for diverse, high-fidelity image generation.

Overview of Rebooted Auxiliary Classifier GANs (ReACGAN)

The paper on Rebooted Auxiliary Classifier GANs (ReACGAN) addresses the known limitations of the Auxiliary Classifier GAN (ACGAN) in the context of conditional Generative Adversarial Networks (cGANs). ACGAN has been extensively used due to its simplicity and ability to incorporate class information in image generation. However, its stability issues, particularly as the number of classes grows, and its tendency to generate homogenous samples, necessitated this research into more robust methodologies.

Key Contributions

1. Understanding Gradient Instability: The paper identifies that the gradient explosion in ACGAN's classifier contributes significantly to early training collapse. This is particularly problematic as the number of classes in a dataset increases, leading ACGAN to focus excessively on classifying images rather than learning meaningful features.
2. Feature Normalization Solution: A foundational solution proposed is the normalization of feature vectors, projecting them onto a unit hypersphere. This intervention is shown to stabilize training by controlling the internal dynamics of the model, effectively mitigating gradient instabilities.
3. Introduction of Data-to-Data Cross-Entropy Loss (D2D-CE): The D2D-CE is a novel loss function designed to take advantage of relational data-to-data information beyond standard data-to-class relationships. It incorporates margin values to enhance intra-class variation and inter-class separability, creating a training process that capitalizes on complex data relationships while minimizing easy sample influence.
4. Empirical Validation and Results: ReACGAN demonstrates superior performance across CIFAR10, Tiny-ImageNet, CUB200, and ImageNet datasets, achieving state-of-the-art results. The improvements in Fréchet Inception Distance (FID) and Precision-Recall metrics underline its capability for generating both diverse and high-fidelity images.

Implications and Future Directions

Theoretical implications of ReACGAN indicate significant advances in understanding and stabilizing GAN training, particularly through addressing gradient explosion issues. Practically, its applicability in scenarios requiring robust class-conditioned image generation is promising. ReACGAN's incorporation of advanced margin techniques and its harmonization with architectures like StyleGAN2 suggest it could significantly influence future AI developments in generative modeling.

Potential future directions include further exploration of ReACGAN within diverse architectural settings and its applicability to other domains beyond image generation. Investigating the fusion of ReACGAN with other advanced augmentation techniques or novel loss functions might provide additional robustness and efficacy in varied data environments. The insights into gradient stabilization could also inform improvements in training stability for other deep learning models.

Overall, ReACGAN represents a significant step forward in the ongoing refinement of GAN-based models, addressing critical issues that have hindered their full potential in the past.