Online Multi-Granularity Distillation for GAN Compression (2108.06908v2)

Abstract: Generative Adversarial Networks (GANs) have witnessed prevailing success in yielding outstanding images, however, they are burdensome to deploy on resource-constrained devices due to ponderous computational costs and hulking memory usage. Although recent efforts on compressing GANs have acquired remarkable results, they still exist potential model redundancies and can be further compressed. To solve this issue, we propose a novel online multi-granularity distillation (OMGD) scheme to obtain lightweight GANs, which contributes to generating high-fidelity images with low computational demands. We offer the first attempt to popularize single-stage online distillation for GAN-oriented compression, where the progressively promoted teacher generator helps to refine the discriminator-free based student generator. Complementary teacher generators and network layers provide comprehensive and multi-granularity concepts to enhance visual fidelity from diverse dimensions. Experimental results on four benchmark datasets demonstrate that OMGD successes to compress 40x MACs and 82.5X parameters on Pix2Pix and CycleGAN, without loss of image quality. It reveals that OMGD provides a feasible solution for the deployment of real-time image translation on resource-constrained devices. Our code and models are made public at: https://github.com/bytedance/OMGD.

• The paper introduces an online single-stage distillation framework that compresses GANs without traditional multi-stage processes or reliance on discriminators.
• It employs multi-granularity techniques by leveraging diverse teacher architectures to transfer layered knowledge to a compact student generator.
• Empirical results demonstrate significant compression, reducing MACs by up to 46.6× and parameters by 82.5× while maintaining image quality and real-time performance.

Online Multi-Granularity Distillation for GAN Compression

The paper "Online Multi-Granularity Distillation for GAN Compression" addresses the significant challenge of deploying Generative Adversarial Networks (GANs) on resource-constrained devices. GANs are known for their high-quality image generation capabilities but often suffer from heavy computational and memory requirements. The authors propose an innovative approach, termed Online Multi-Granularity Distillation (OMGD), to compress GAN models effectively, specifically targeting conditional GANs like Pix2Pix and CycleGAN.

Key Contributions

1. Online Distillation Framework: The paper introduces a novel online single-stage distillation strategy tailored for GANs. Unlike traditional multi-stage GAN compression methods, OMGD integrates the distillation process in an end-to-end manner without the dependency on a discriminator or direct ground truth data.
2. Multi-Granularity Concepts: OMGD employs multiple teacher generators with complementary architectures—wider and deeper structures—to provide diverse guidance to the student generator. This multi-granularity approach also includes transferring knowledge from various layers of the teacher model, enhancing the richness and depth of the learned features.
3. Substantial Compression Rates: Empirical results across four benchmark datasets demonstrate significant reductions in computational complexity—up to 46.6$\times$ fewer MACs and 82.5$\times$ reductions in parameters—without compromising the quality of the output images.

Results and Performance

The paper provides comprehensive experimental results comparing the proposed method with existing GAN compression techniques. Key numerical outcomes include:

• Pix2Pix and CycleGAN Models: OMGD achieves substantial reductions in model size while maintaining, or even improving, the image fidelity. The approach shows a marked improvement in metrics such as FID (Frechet Inception Distance) and mIoU (mean Intersection over Union) on popular datasets.
• Real-Time Deployment: The OMGD framework effectively reduces latency on mobile devices by up to 90%, signaling its practical viability for real-time applications.

Implications and Future Directions

The implications of this research are substantial for both academic and industrial applications, particularly in areas like image synthesis and translation where GANs are prevalent. By enabling efficient deployment of GANs on edge devices, this work opens the door for widespread use of advanced generative models in mobile and IoT environments.

Future work could explore further refinement of the distillation process, potentially integrating adaptive methods that dynamically balance the depth and width according to specific tasks or datasets. Additionally, expanding the technique to cover other types of GANs or probabilistic models could further enhance its applicability.

In summary, the paper presents a well-substantiated advancement in GAN compression, providing a robust framework that balances computational efficiency with quality preservation. This contributes meaningfully to the ongoing development of scalable and deployable AI models.