Training GANs with Optimism (1711.00141v2)

Abstract: We address the issue of limit cycling behavior in training Generative Adversarial Networks and propose the use of Optimistic Mirror Decent (OMD) for training Wasserstein GANs. Recent theoretical results have shown that optimistic mirror decent (OMD) can enjoy faster regret rates in the context of zero-sum games. WGANs is exactly a context of solving a zero-sum game with simultaneous no-regret dynamics. Moreover, we show that optimistic mirror decent addresses the limit cycling problem in training WGANs. We formally show that in the case of bi-linear zero-sum games the last iterate of OMD dynamics converges to an equilibrium, in contrast to GD dynamics which are bound to cycle. We also portray the huge qualitative difference between GD and OMD dynamics with toy examples, even when GD is modified with many adaptations proposed in the recent literature, such as gradient penalty or momentum. We apply OMD WGAN training to a bioinformatics problem of generating DNA sequences. We observe that models trained with OMD achieve consistently smaller KL divergence with respect to the true underlying distribution, than models trained with GD variants. Finally, we introduce a new algorithm, Optimistic Adam, which is an optimistic variant of Adam. We apply it to WGAN training on CIFAR10 and observe improved performance in terms of inception score as compared to Adam.

• The paper presents the use of Optimistic Mirror Descent to achieve last-iterate convergence in GAN training.
• It demonstrates theoretical advantages in zero-sum games with empirical validation on bioinformatics and CIFAR10 tasks.
• The research introduces Optimistic Adam, yielding improved inception scores as a robust alternative to gradient descent.

Training GANs with Optimism: A Summary

The paper "Training GANs with Optimism" by Daskalakis et al. addresses the critical issue of instability during the training of Generative Adversarial Networks (GANs) by proposing Optimistic Mirror Descent (OMD) as a robust alternative to the conventional gradient descent (GD) methods.

GANs, widely recognized for their capacity to model complex data distributions, encounter significant challenges in training stability. A considerable problem is the occurrence of limit cycling, where the training process oscillates without convergence. This paper introduces OMD for training Wasserstein GANs (WGANs) to tackle these prevalent cycles.

Methodology and Theoretical Contributions

OMD is an advanced variant of GD characterized by faster convergence rates and enhanced stability in the context of zero-sum games. The authors demonstrate that OMD diverges from the typical GD approach by incorporating predictive components into the gradient update, thus minimizing regret more efficiently.

A pivotal theoretical advancement presented is that, for bi-linear zero-sum games, OMD ensures last-iterate convergence to an equilibrium, contrary to the GD's tendency to cycle indefinitely. This result underscores OMD’s potential in training scenarios where stability and convergence are critical.

Experimental Evaluation

The paper substantiates its theoretical propositions with empirical evidence through practical applications. OMD is applied to the domain of bioinformatics for generating DNA sequences, revealing smaller KL divergence compared to GD variants—a strong indicator of the model's fitness to the true distribution.

Moreover, the introduction of Optimistic Adam, an augmentation of the widely used Adam optimizer, demonstrates its efficacy by yielding improved inception scores on CIFAR10, a benchmark dataset for image generation tasks.

Implications and Future Directions

The implications of adopting OMD extend beyond merely mitigating cycle issues in GAN training. Faster convergence rates and stable last-iterate properties suggest broader applicability across machine learning domains involving adversarial setups. The proposed methodology equips researchers and practitioners with a tool to refine existing GAN frameworks, optimizing both training speed and model performance.

The promising results from OMD and Optimistic Adam pave the way for future research in refining these techniques for diverse and more complex GAN architectures and other zero-sum game contexts. There remains substantial scope for exploring how these approaches can be tailored and scaled, especially in environments with non-convex landscapes.

In conclusion, this paper provides a significant contribution to the methodologies for training GANs by deploying optimistic algorithmic strategies. This research opens avenues for enhancing the reliability and efficiency of GANs, thereby expanding their potential applications in artificial intelligence.