Conditional Image Synthesis With Auxiliary Classifier GANs (1610.09585v4)

Abstract: Synthesizing high resolution photorealistic images has been a long-standing challenge in machine learning. In this paper we introduce new methods for the improved training of generative adversarial networks (GANs) for image synthesis. We construct a variant of GANs employing label conditioning that results in 128x128 resolution image samples exhibiting global coherence. We expand on previous work for image quality assessment to provide two new analyses for assessing the discriminability and diversity of samples from class-conditional image synthesis models. These analyses demonstrate that high resolution samples provide class information not present in low resolution samples. Across 1000 ImageNet classes, 128x128 samples are more than twice as discriminable as artificially resized 32x32 samples. In addition, 84.7% of the classes have samples exhibiting diversity comparable to real ImageNet data.

• The paper demonstrates that integrating label conditioning with GANs enhances training stability and image quality.
• It employs a dual-objective framework where the discriminator predicts both image source and class label to optimize the generator.
• The study shows that direct 128×128 image synthesis preserves discriminability and diversity, with 84.7% of classes matching real data diversity.

Conditional Image Synthesis with Auxiliary Classifier GANs

The paper "Conditional Image Synthesis with Auxiliary Classifier GANs" by Augustus Odena, Christopher Olah, and Jonathon Shlens introduces methodologies to enhance the training of generative adversarial networks (GANs) for image synthesis, specifically through the construction of an Auxiliary Classifier GAN (AC-GAN). The primary contributions of this work include the integration of label conditioning into GANs and the demonstration of an image synthesis model trained on all 1000 classes of the ImageNet dataset at a resolution of 128×128 pixels.

AC-GAN Architecture and Training

In an AC-GAN, each generated image is associated with a class label in addition to the random noise vector typically used in GANs. The generator (G) uses both the noise vector (z) and the class label (c) to produce an image, while the discriminator (D) outputs both a source probability (indicating whether an image is real or fake) and a class probability. This dual-objective framework helps improve the stability of the GAN training process.

The objective function for AC-GANs consists of two components: the log-likelihood of the correct image source (real or fake) and the log-likelihood of the correct class label. The generator is trained to maximize the likelihood of the correct class while minimizing the likelihood of the image being classified as fake. This structure leads to enhanced image synthesis quality and stability.

Image Synthesis Performance and Assessment

The paper introduces novel metrics for evaluating the discriminability and diversity of generated images. Discriminability is measured using a pre-trained Inception network to classify synthesized images, with findings indicating higher resolution (128×128) images are more discriminable than lower resolutions. In particular, downsampling the synthesized 128×128 images to 32×32 resolution results in a 50% reduction in discriminability, thereby underscoring the importance of generating high-resolution images directly.

Diversity is quantified using the multi-scale structural similarity index (MS-SSIM), a perceptual similarity measure. The MS-SSIM scores between pairs of GAN-generated images within the same class are used as a proxy for perceptual diversity. The paper indicates that 84.7% of the classes exhibit diversity comparable to real ImageNet data.

Practical and Theoretical Implications

This research demonstrates that by conditioning GANs with auxiliary classifiers, it is feasible to generate visually coherent and high-resolution images across a large number of classes. Such advancements have significant practical implications for various applications, including but not limited to image generation for data augmentation, semi-supervised learning, and potentially enhancing performance in computer vision tasks where labeled data might be scarce.

Theoretically, the success of AC-GANs in maintaining both high discriminability and diversity of generated images challenges some prevailing assumptions about GANs, particularly the notion that increasing sample quality necessarily leads to mode collapse. The empirical results suggest that better structured latent spaces can harmonize sample quality and variability, paving the way for more robust generative models.

Future Directions

Several future research directions are identified, motivated by the current paper's findings and limitations:

1. Enhancing Discriminability: Despite improvements, the average classification accuracy of synthesized images remains significantly below that of real images. Integrating fixed pre-trained models into the discriminator network could further bolster the discriminability.
2. Improving Training Stability: The reliance on class splitting to enhance training stability suggests a need for models capable of handling higher diversity within unsegmented datasets.
3. Semi-Supervised Learning: The AC-GAN model offers potential for advancing semi-supervised learning paradigms, exploiting the rich priors over natural image statistics constructed by generative models.

Conclusion

The paper by Odena et al. provides integral contributions to the field of generative adversarial networks through the introduction of AC-GANs. By demonstrating the ability to produce high-quality, high-resolution images across numerous classes, and providing robust methods for evaluating image synthesis models, this research marks a significant step forward in GAN development. The implications for both practical applications and theoretical understanding of GANs are profound, with ample avenues for further exploration and refinement.