Image Augmentations for GAN Training (2006.02595v1)

Abstract: Data augmentations have been widely studied to improve the accuracy and robustness of classifiers. However, the potential of image augmentation in improving GAN models for image synthesis has not been thoroughly investigated in previous studies. In this work, we systematically study the effectiveness of various existing augmentation techniques for GAN training in a variety of settings. We provide insights and guidelines on how to augment images for both vanilla GANs and GANs with regularizations, improving the fidelity of the generated images substantially. Surprisingly, we find that vanilla GANs attain generation quality on par with recent state-of-the-art results if we use augmentations on both real and generated images. When this GAN training is combined with other augmentation-based regularization techniques, such as contrastive loss and consistency regularization, the augmentations further improve the quality of generated images. We provide new state-of-the-art results for conditional generation on CIFAR-10 with both consistency loss and contrastive loss as additional regularizations.

Evaluation of Image Augmentations in GAN Training

The paper "Image Augmentations for GAN Training" investigates the impact of image augmentation techniques on the training of Generative Adversarial Networks (GANs), which are fundamental models for generating realistic images. The paper systematically evaluates a broad array of augmentation strategies to enhance the fidelity of the images generated by GANs, providing valuable insights and guidelines for both vanilla GANs and GANs with regularizations.

Key Contributions

1. Comprehensive Evaluation of Image Augmentations: The paper conducts an extensive analysis of commonly used image augmentation techniques. It evaluates these techniques' efficacy and robustness in improving GANs' performance, specifically focusing on both real and generated images during the training process.
2. Augmenting Generated Samples: One of the pivotal findings is that augmenting both real and synthesized images improves GAN performance significantly compared to augmenting only real images. This dual strategy allows vanilla GANs to achieve image generation quality comparable to state-of-the-art models enhanced by consistency regularization.
3. Incorporation with Regularizations: The research further examines combining data augmentations with regularization strategies, such as contrastive loss and consistency regularization. These augmentations improve the Fréchet Inception Distance (FID) score, indicating higher image fidelity.
4. State-of-the-Art Results on CIFAR-10: By integrating the optimal augmentation strategy with consistency and contrastive losses, the paper improves the FID score for conditional image generation on the CIFAR-10 dataset from 9.21 to 8.30.

Methodology

The systematic investigation involves both basic and advanced augmentation operations such as cropping, flipping, color jittering, and more complex techniques like MixUp and CutMix. The augmentations are applied to the CIFAR-10 dataset using widely recognized GAN architectures, SNDCGAN for unconditional generation and BigGAN for conditional generation.

The evaluation metric, FID, is utilized to monitor the quality of generated images and assess the impact of various augmentation techniques. Additionally, the interaction between augmentation strategies and different regularization approaches, such as contrastive and consistency losses, is explored.

Implications and Future Directions

The findings have significant implications for the development and training of GANs. Incorporating robust augmentation techniques can lead to substantial improvements in image synthesis quality, offering a promising direction for subsequent research in the domain of generative models.

Future work could focus on extending these insights by exploring automatic augmentation strategies that dynamically adjust augmentation techniques during training. Additionally, the interplay between different augmentation strategies can be analyzed to explore potential synergistic effects, leading to even more effective GAN training protocols.

The conclusions of this research provide a foundation for developing more sophisticated and effective augmentation strategies, advancing the field toward greater realism in computer-generated images without necessitating complex and computationally intensive regularization techniques.