Omni-GAN: On the Secrets of cGANs and Beyond (2011.13074v3)

Abstract: The conditional generative adversarial network (cGAN) is a powerful tool of generating high-quality images, but existing approaches mostly suffer unsatisfying performance or the risk of mode collapse. This paper presents Omni-GAN, a variant of cGAN that reveals the devil in designing a proper discriminator for training the model. The key is to ensure that the discriminator receives strong supervision to perceive the concepts and moderate regularization to avoid collapse. Omni-GAN is easily implemented and freely integrated with off-the-shelf encoding methods (e.g., implicit neural representation, INR). Experiments validate the superior performance of Omni-GAN and Omni-INR-GAN in a wide range of image generation and restoration tasks. In particular, Omni-INR-GAN sets new records on the ImageNet dataset with impressive Inception scores of 262.85 and 343.22 for the image sizes of 128 and 256, respectively, surpassing the previous records by 100+ points. Moreover, leveraging the generator prior, Omni-INR-GAN can extrapolate low-resolution images to arbitrary resolution, even up to x60+ higher resolution. Code is available.

• The paper introduces Omni-GAN, a novel variant that optimizes discriminator design to effectively mitigate mode collapse.
• It integrates with encoding methods like implicit neural representation, broadening applications in image generation and restoration.
• Experiments on ImageNet showcase state-of-the-art Inception scores and remarkable high-resolution upscaling capabilities.

"Omni-GAN: On the Secrets of cGANs and Beyond" addresses the critical issue of performance bottlenecks and mode collapse in conditional generative adversarial networks (cGANs), which are popular for generating high-fidelity images. The authors propose a novel variant named Omni-GAN that dramatically improves the robustness and quality of image generation.

The paper explores the importance of a well-designed discriminator in the training process of cGANs. It posits that effective supervision of the discriminator is paramount, ensuring it can correctly perceive various concepts within the generated images, while also applying moderate regularization techniques to prevent mode collapse.

Key innovations of Omni-GAN include:

1. Discriminator Design: The authors emphasize balancing discriminator supervision and regularization. Strong supervision helps the discriminator understand intricate details, while regularization prevents it from becoming overly confident, thus mitigating the risk of mode collapse.
2. Integration with Encoding Methods: Omni-GAN can seamlessly integrate with existing encoding methods such as implicit neural representation (INR). This adaptability makes it versatile for a range of image-related tasks, from generation to restoration.
3. Performance Metrics: Experiments demonstrate that Omni-GAN, along with its variant Omni-INR-GAN, achieves state-of-the-art performance. Notably, Omni-INR-GAN sets new benchmarks on the challenging ImageNet dataset, recording phenomenal Inception scores of 262.85 and 343.22 for image resolutions of 128 and 256, respectively. These scores surpass previous records by over 100 points, showcasing the substantial improvements facilitated by their approach.
4. High-Resolution Extrapolation: Another standout feature of Omni-INR-GAN is its generator prior, which enables it to upscale low-resolution images to exceptionally high resolutions. This includes performing upscaling tasks to resolutions x60+ times greater than the original, all while maintaining image quality.

The paper’s contributions underscore the potential of a meticulously designed discriminator in enhancing cGAN performance. By addressing the dual challenges of supervision and regularization, Omni-GAN achieves impressive results in both image generation and restoration, setting a new high watermark in the field. The availability of code also encourages further exploration and application of their findings.