Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold (2312.14776v1)

Published 22 Dec 2023 in cs.CV and eess.IV

Abstract: Generative Adversarial Networks (GANs) have shown remarkable success in modeling complex data distributions for image-to-image translation. Still, their high computational demands prohibit their deployment in practical scenarios like edge devices. Existing GAN compression methods mainly rely on knowledge distillation or convolutional classifiers' pruning techniques. Thus, they neglect the critical characteristic of GANs: their local density structure over their learned manifold. Accordingly, we approach GAN compression from a new perspective by explicitly encouraging the pruned model to preserve the density structure of the original parameter-heavy model on its learned manifold. We facilitate this objective for the pruned model by partitioning the learned manifold of the original generator into local neighborhoods around its generated samples. Then, we propose a novel pruning objective to regularize the pruned model to preserve the local density structure over each neighborhood, resembling the kernel density estimation method. Also, we develop a collaborative pruning scheme in which the discriminator and generator are pruned by two pruning agents. We design the agents to capture interactions between the generator and discriminator by exchanging their peer's feedback when determining corresponding models' architectures. Thanks to such a design, our pruning method can efficiently find performant sub-networks and can maintain the balance between the generator and discriminator more effectively compared to baselines during pruning, thereby showing more stable pruning dynamics. Our experiments on image translation GAN models, Pix2Pix and CycleGAN, with various benchmark datasets and architectures demonstrate our method's effectiveness.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. Compressing gans using knowledge distillation. arXiv preprint arXiv:1902.00159.
  2. Do Deep Nets Really Need to be Deep? In NeurIPS.
  3. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432.
  4. Adversarial learning with local coordinate coding. In International Conference on Machine Learning, 707–715.
  5. Unsupervised learning of visual features by contrasting cluster assignments. Advances in Neural Information Processing Systems, 33: 9912–9924.
  6. Instance-Conditioned GAN. In Advances in Neural Information Processing Systems.
  7. Everybody dance now. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 5933–5942.
  8. TinyGAN: Distilling BigGAN for Conditional Image Generation. In Proceedings of the Asian Conference on Computer Vision.
  9. Distilling portable generative adversarial networks for image translation. In Proceedings of the AAAI Conference on Artificial Intelligence.
  10. A simple framework for contrastive learning of visual representations. In International conference on machine learning.
  11. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297.
  12. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP 2014.
  13. Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs. In International Conference on Learning Representations.
  14. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3213–3223.
  15. Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks. In International Conference on Learning Representations.
  16. AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks. In ICML.
  17. Interpretations steered network pruning via amortized inferred saliency maps. In European Conference on Computer Vision, 278–296. Springer.
  18. EffConv: Efficient Learning of Kernel Sizes for Convolution Layers of CNNs. In Thirty-Seventh AAAI Conference on Artificial Intelligence. AAAI Press.
  19. Adversarialnas: Adversarial neural architecture search for gans. In CVPR, 5680–5689.
  20. A Survey on Efficient Convolutional Neural Networks and Hardware Acceleration. Electronics, 11(6): 945.
  21. Autogan: Neural architecture search for generative adversarial networks. In ICCV, 3224–3234.
  22. Generative adversarial nets. NeurIPS, 27.
  23. Bootstrap your own latent-a new approach to self-supervised learning. NeurIPS.
  24. Gumbel, E. J. 1954. Statistical theory of extreme values and some practical applications: a series of lectures, volume 33. US Government Printing Office.
  25. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems.
  26. Ganspace: Discovering interpretable gan controls. Advances in Neural Information Processing Systems.
  27. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  28. Amc: Automl for model compression and acceleration on mobile devices. In ECCV.
  29. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30.
  30. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
  31. Slimmable Generative Adversarial Networks. In AAAI. AAAI Press.
  32. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
  33. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV.
  34. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, 448–456. PMLR.
  35. Image-to-image translation with conditional adversarial networks. In CVPR, 1125–1134.
  36. Categorical Reparameterization with Gumbel-Softmax. In ICLR 2017.
  37. Teachers Do More Than Teach: Compressing Image-to-Image Models. In CVPR.
  38. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, 694–711. Springer.
  39. Disconnected manifold learning for generative adversarial networks. NeurIPS, 31.
  40. Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs. arXiv preprint arXiv:2206.14658.
  41. Adam: A Method for Stochastic Optimization. In Bengio, Y.; and LeCun, Y., eds., 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  42. Photo-realistic single image super-resolution using a generative adversarial network. In CVPR.
  43. Pruning filters for efficient convnets. ICLR.
  44. Gan compression: Efficient architectures for interactive conditional gans. In CVPR.
  45. Learning efficient gans for image translation via differentiable masks and co-attention distillation. IEEE Transactions on Multimedia.
  46. Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme. Advances in Neural Information Processing Systems, 34.
  47. Anycost gans for interactive image synthesis and editing. In CVPR.
  48. Learning efficient convolutional networks through network slimming. In ICCV.
  49. Manifold Learning Benefits GANs. In CVPR.
  50. Tensor Component Analysis for Interpreting the Latent Space of GANs. In BMVC.
  51. Semantic image synthesis with spatially-adaptive normalization. In CVPR.
  52. Parzen, E. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3).
  53. Xnor-net: Imagenet classification using binary convolutional neural networks. In ECCV. Springer.
  54. Online multi-granularity distillation for gan compression. In ICCV.
  55. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, 234–241. Springer.
  56. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR.
  57. Closed-form factorization of latent semantics in gans. In CVPR.
  58. Co-evolutionary compression for unpaired image translation. In ICCV.
  59. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929–1958.
  60. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, 6105–6114. PMLR.
  61. A global geometric framework for nonlinear dimensionality reduction. science, 290(5500): 2319–2323.
  62. Grafit: Learning fine-grained image representations with coarse labels. In ICCV.
  63. WarpedGANSpace: Finding non-linear RBF paths in GAN latent space. In ICCV.
  64. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022.
  65. Matching networks for one shot learning. Advances in neural information processing systems, 29.
  66. Unsupervised discovery of interpretable directions in the gan latent space. In International conference on machine learning.
  67. Fbnetv2: Differentiable neural architecture search for spatial and channel dimensions. In CVPR.
  68. GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework. In ECCV.
  69. Coarse-to-Fine Searching for Efficient Generative Adversarial Networks. arXiv preprint arXiv:2104.09223.
  70. Video-to-video synthesis. arXiv preprint arXiv:1808.06601.
  71. Esrgan: Enhanced super-resolution generative adversarial networks. In ECCV workshops.
  72. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search. In CVPR.
  73. Good subnetworks provably exist: Pruning via greedy forward selection. In ICML.
  74. Fine-grained visual comparisons with local learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 192–199.
  75. Self-supervised generative adversarial compression. NeurIPS, 33.
  76. Region-aware Knowledge Distillation for Efficient Image-to-Image Translation. arXiv preprint arXiv:2205.12451.
  77. Wavelet knowledge distillation: Towards efficient image-to-image translation. In CVPR.
  78. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now