Mode Regularized Generative Adversarial Networks (1612.02136v5)

Abstract: Although Generative Adversarial Networks achieve state-of-the-art results on a variety of generative tasks, they are regarded as highly unstable and prone to miss modes. We argue that these bad behaviors of GANs are due to the very particular functional shape of the trained discriminators in high dimensional spaces, which can easily make training stuck or push probability mass in the wrong direction, towards that of higher concentration than that of the data generating distribution. We introduce several ways of regularizing the objective, which can dramatically stabilize the training of GAN models. We also show that our regularizers can help the fair distribution of probability mass across the modes of the data generating distribution, during the early phases of training and thus providing a unified solution to the missing modes problem.

• The paper introduces novel mode regularizers that stabilize GAN training by mitigating mode collapse and ensuring balanced probability distribution.
• It presents a new MDGAN framework that splits the generative process into matching and diffusion steps, leading to improved MODE scores on datasets like MNIST and CelebA.
• Empirical results demonstrate enhanced sample diversity and reduced missing modes, outperforming benchmarks such as VAEGAN and DCGAN in qualitative and quantitative evaluations.

Mode Regularized Generative Adversarial Networks

The paper "Mode Regularized Generative Adversarial Networks" addresses a prominent issue in the training of Generative Adversarial Networks (GANs): the instability and mode collapse during the generative process. GANs, although achieving state-of-the-art outcomes in various tasks like image generation and video prediction, are susceptible to generating samples from only a few modes of the data distribution. This can result in reduced diversity and entropy within generated samples notably missing smaller modes.

Main Contributions

The authors propose a novel framework incorporating mode regularizers to stabilize GAN training and enhance the diversity of generated samples:

1. Issue Identification: The instability and mode collapse in GANs are attributed to the discriminator's decision boundaries, which can misdirect probability mass across the feature space.
2. Regularizer Design: The paper introduces regularizers for the GAN objective, prominently using geometric metric regularizers and mode regularizers. These regularizers are intended to provide stable gradients and equitably distribute probability mass across all modes of the data distribution.
3. Training Framework: A new training algorithm called Manifold-Diffusion Training (MDGAN) is implemented to separate the generative process into matching and diffusion steps, improving the stability and quality of the GAN's output.

Results and Analysis

Significant outcomes from empirical evaluations highlight the effectiveness of the proposed methods:

• On the MNIST dataset, the use of mode regularizers demonstrated notable improvements in both the stability of training and the quality of generated samples, reflected in higher MODE scores compared to traditional GANs.
• In complex settings like the CelebA dataset, MDGAN achieved a substantial reduction in missing modes, demonstrating improved adherence to the diversity present in the real-world data distribution.
• Qualitative analysis shows that samples from the proposed method exhibit fewer distortions and enhanced diversity, particularly compared to other advanced models such as VAEGAN and DCGAN.

Implications and Future Directions

The introduction of regularizers offers a promising path to address GAN training challenges, specifically mode collapse and instability. The proposed regularizers effectively balance model variance and bias, suggesting that further exploration into other forms of regularization could be fruitful. Additionally, future research might investigate the applicability of these regularizers to other variants of GANs, and how they influence different tasks such as sequence generation or 3D modeling.

The discussion in this paper contributes a significant methodological advancement in how GANs may be trained more effectively, paving the way for more robust and diverse generative models in the artificial intelligence domain.