Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Mode-Aware Continual Learning for Conditional Generative Adversarial Networks (2305.11400v3)

Published 19 May 2023 in cs.LG and stat.ML

Abstract: The main challenge in continual learning for generative models is to effectively learn new target modes with limited samples while preserving previously learned ones. To this end, we introduce a new continual learning approach for conditional generative adversarial networks by leveraging a mode-affinity score specifically designed for generative modeling. First, the generator produces samples of existing modes for subsequent replay. The discriminator is then used to compute the mode similarity measure, which identifies a set of closest existing modes to the target. Subsequently, a label for the target mode is generated and given as a weighted average of the labels within this set. We extend the continual learning model by training it on the target data with the newly-generated label, while performing memory replay to mitigate the risk of catastrophic forgetting. Experimental results on benchmark datasets demonstrate the gains of our continual learning approach over the state-of-the-art methods, even when using fewer training samples.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. The dynamic distance between learning tasks: From kolmogorov complexity to transfer learning via quantum physics and the information bottleneck of the weights of deep networks. NeurIPS Workshop on Integration of Deep Learning Theories, 2018.
  2. Task2Vec: Task Embedding for Meta-Learning. arXiv e-prints, art. arXiv:1902.03545, Feb. 2019.
  3. Causal knowledge transfer from task affinity. arXiv preprint arXiv:2210.00380, 2022.
  4. Wasserstein generative adversarial networks. In International conference on machine learning, pp.  214–223. PMLR, 2017.
  5. Big self-supervised models advance medical image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  3478–3488, 2021.
  6. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
  7. A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer vision, graphics, and image processing, 37(1):54–115, 1987.
  8. Coupled end-to-end transfer learning with generalized fisher information. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.  4329–4338, 2018.
  9. Memory replay gans: Learning to generate images from new categories without forgetting [c]. In The 32nd International Conference on Neural Information Processing Systems, Montréal, Canada, pp.  5966–5976, 2018.
  10. Effectively unbiased fid and inception score and where to find them. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  6070–6079, 2020.
  11. Gan memory with no forgetting. Advances in Neural Information Processing Systems, 33:16481–16494, 2020.
  12. Large scale fine-grained categorization and domain-specific transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  4109–4118, 2018.
  13. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  14. K. Dwivedi and G. Roig. Representation similarity analysis for efficient task taxonomy and transfer learning. In CVPR. IEEE Computer Society, 2019.
  15. A conditional gan-based approach for enhancing transfer learning performance in few-shot hcr tasks. Scientific Reports, 12(1):16271, 2022.
  16. Deep spatial autoencoders for visuomotor learning. In Robotics and Automation (ICRA), 2016 IEEE International Conference on, pp.  512–519. IEEE, 2016.
  17. Borrowing treasures from the wealthy: Deep transfer learning through selective joint fine-tuning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  1086–1095, 2017.
  18. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017.
  19. Spottune: transfer learning through adaptive fine-tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  4805–4814, 2019.
  20. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
  21. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
  22. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  328–339, 2018.
  23. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
  24. Learning multiple layers of features from tiny images. Citeseer, 2009.
  25. Supervised encoding for discrete representation learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.  3447–3451. IEEE, 2020.
  26. Improved automated machine learning from transfer learning. arXiv e-prints, pp.  arXiv–2103, 2021a.
  27. Task-aware neural architecture search. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.  4090–4094. IEEE, 2021b.
  28. Fisher task distance and its application in neural architecture search. IEEE Access, 10:47235–47249, 2022a.
  29. Improving open-domain dialogue evaluation with a causal inference model. Diversity in Dialogue Systems: 13th International Workshop on Spoken Dialogue System Technology (IWSDS), 2023.
  30. Task affinity with maximum bipartite matching in few-shot learning. In International Conference on Learning Representations, 2022b.
  31. Mnist handwritten digit database. AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist, 2:18, 2010.
  32. An improved evaluation framework for generative adversarial networks. arXiv preprint arXiv:1803.07474, 2018.
  33. Label efficient learning of transferable representations acrosss domains and tasks. In Advances in Neural Information Processing Systems, pp.  164–176, 2017.
  34. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.  7765–7773, 2018.
  35. Ternary feature masks: continual learning without any forgetting. arXiv preprint arXiv:2001.08714, 4(5):6, 2020.
  36. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp.  109–165. Elsevier, 1989.
  37. Mapping and revising markov logic networks for transfer learning. In AAAI, volume 7, pp.  608–614, 2007.
  38. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.
  39. Inductive transfer for bayesian network structure learning. In Artificial Intelligence and Statistics, pp.  339–346, 2007.
  40. Automated flower classification over a large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp.  722–729. IEEE, 2008.
  41. OpenAI. Gpt-3.5. Computer software, 2021. URL https://openai.com/blog/gpt-3-5/.
  42. Zero-shot task transfer, 2019.
  43. S. J. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345–1359, Oct 2010. ISSN 1041-4347. doi: 10.1109/TKDE.2009.191.
  44. Random path selection for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
  45. itaml: An incremental task-agnostic meta-learning approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  13588–13597, 2020.
  46. Cnn features off-the-shelf: An astounding baseline for recognition. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW ’14, pp.  512–519, Washington, DC, USA, 2014. IEEE Computer Society. ISBN 978-1-4799-4308-1. doi: 10.1109/CVPRW.2014.131. URL http://dx.doi.org.stanford.idm.oclc.org/10.1109/CVPRW.2014.131.
  47. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.  2001–2010, 2017.
  48. Closed-loop memory gan for continual learning. arXiv preprint arXiv:1811.01146, 2018.
  49. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146, 1995.
  50. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
  51. Continual learning in generative adversarial nets. arXiv preprint arXiv:1705.08395, 2017.
  52. Guest editor’s introduction: special issue on inductive transfer learning. Machine Learning, 73(3):215–220, 2008.
  53. Calibrating cnns for lifelong learning. Advances in Neural Information Processing Systems, 33:15579–15590, 2020.
  54. Which tasks should be learned together in multi-task learning? In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.  9120–9132. PMLR, 13–18 Jul 2020a. URL http://proceedings.mlr.press/v119/standley20a.html.
  55. Which tasks should be learned together in multi-task learning? In International Conference on Machine Learning, pp.  9120–9132. PMLR, 2020b.
  56. Cam-gan: Continual adaptation modules for generative adversarial networks. Advances in Neural Information Processing Systems, 34:15175–15187, 2021.
  57. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  58. DALL•E: Creating images from text. OpenAI, 2021. URL https://openai.com/dall-e/.
  59. Efficient feature transformations for discriminative and generative continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  13865–13875, 2021.
  60. Neural taskonomy: Inferring the similarity of task-derived representations from brain activity. BioRxiv, pp.  708016, 2019.
  61. Transferring gans: generating images from limited data. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  218–234, 2018.
  62. Mika Westerlund. The emergence of deepfake technology: A review. Technology innovation management review, 9(11), 2019.
  63. Memory replay gans: Learning to generate new categories without forgetting. Advances in Neural Information Processing Systems, 31, 2018.
  64. Ju Xu and Zhanxing Zhu. Reinforced continual learning. Advances in Neural Information Processing Systems, 31, 2018.
  65. Inverse airfoil design method for generating varieties of smooth airfoils using conditional wgan-gp. arXiv preprint arXiv:2110.00212, 2021.
  66. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547, 2017.
  67. Taskonomy: Disentangling task transfer learning. In 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018.
  68. Continual learning through synaptic intelligence. In International conference on machine learning, pp.  3987–3995. PMLR, 2017.
  69. Lifelong gan: Continual learning for conditional image generation. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  2759–2768, 2019.
  70. Piggyback gan: Efficient lifelong learning for image conditioned generation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16, pp.  397–413. Springer, 2020.
  71. Toward multimodal image-to-image translation. Advances in neural information processing systems, 30, 2017.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now