Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GUIDE: Guidance-based Incremental Learning with Diffusion Models (2403.03938v2)

Published 6 Mar 2024 in cs.LG

Abstract: We introduce GUIDE, a novel continual learning approach that directs diffusion models to rehearse samples at risk of being forgotten. Existing generative strategies combat catastrophic forgetting by randomly sampling rehearsal examples from a generative model. Such an approach contradicts buffer-based approaches where sampling strategy plays an important role. We propose to bridge this gap by incorporating classifier guidance into the diffusion process to produce rehearsal examples specifically targeting information forgotten by a continuously trained model. This approach enables the generation of samples from preceding task distributions, which are more likely to be misclassified in the context of recently encountered classes. Our experimental results show that GUIDE significantly reduces catastrophic forgetting, outperforming conventional random sampling approaches and surpassing recent state-of-the-art methods in continual learning with generative replay.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Online continual learning with maximal interfered retrieval. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/15825aee15eb335cc13f9b559f166ee8-Paper.pdf.
  2. Diffusion visual counterfactual explanations. arXiv preprint arXiv:2210.11841, 2022.
  3. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  843–852, 2023.
  4. Il2m: Class incremental learning with dual memory. In International Conference on Computer Vision, 2019.
  5. End-to-end incremental learning. In European Conference on Computer Vision, 2018.
  6. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pp.  532–547, 2018.
  7. Dalle mini, 7 2021.
  8. Remember the past: Distilling datasets into addressable memories for neural networks. Advances in Neural Information Processing Systems, 35:34391–34404, 2022.
  9. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
  10. Class-prototype conditional diffusion model for continual learning with generative replay. arXiv preprint arXiv:2312.06710, 2023.
  11. Diffusion self-guidance for controllable image generation. arXiv preprint arXiv:2306.00986, 2023.
  12. French, R. M. Catastrophic forgetting in connectionist networks. Trends in cog. scie., 1999.
  13. Ddgr: continual learning with deep diffusion-based generative replay. In International Conference on Machine Learning, pp.  10744–10763. PMLR, 2023.
  14. Generative Adversarial Networks. In NeurIPS, 2014.
  15. Explaining and harnessing adversarial examples. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6572.
  16. Grossberg, S. Studies of mind and brain. Boston Studies in the Philosophy of Science, 1982.
  17. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in neural information processing systems, pp.  6626–6637, 2017.
  18. Classifier-free diffusion guidance. arXiv preprint arXiv: Arxiv-2207.12598, 2022.
  19. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  20. Learning a unified classifier incrementally via rebalancing. In International Conference on Computer Vision, 2019.
  21. Selective experience replay for lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  22. Gradient-based editing of memory examples for online task-free continual learning. Advances in Neural Information Processing Systems, 34:29193–29205, 2021.
  23. Fearnet: Brain-inspired model for incremental learning. International Conference on Learning Representations, 2018.
  24. Auto-Encoding Variational Bayes. In ICLR, 2014.
  25. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
  26. Krizhevsky, A. Learning multiple layers of features from tiny images. Master’s thesis, University of Tront, 2009.
  27. Retrospective adversarial replay for continual learning. Advances in Neural Information Processing Systems, 35:28530–28544, 2022.
  28. Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems, 32, 2019.
  29. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017.
  30. Generative feature replay for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.  226–227, 2020a.
  31. Mnemonics training: Multi-class incremental learning without forgetting. In Conference on Computer Vision and Pattern Recognition, 2020b.
  32. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30:6467–6476, 2017.
  33. Packnet: Adding multiple tasks to a single network by iterative pruning. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pp.  7765–7773. Computer Vision Foundation / IEEE Computer Society, 2018. doi: 10.1109/CVPR.2018.00810. URL http://openaccess.thecvf.com/content_cvpr_2018/html/Mallya_PackNet_Adding_Multiple_CVPR_2018_paper.html.
  34. Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. (eds.), Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part IV, volume 11208 of Lecture Notes in Computer Science, pp.  72–88. Springer, 2018. doi: 10.1007/978-3-030-01225-0_5. URL https://doi.org/10.1007/978-3-030-01225-0_5.
  35. Continual learning of diffusion models with generative distillation. arXiv preprint arXiv:2311.14028, 2023.
  36. Variational continual learning. In International Conference on Learning Representations, 2018.
  37. Gdumb: A simple approach that questions our progress in continual learning. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  524–540. Springer, 2020.
  38. Lifelong generative modeling. Neurocomputing, 404:381–400, 2020. ISSN 0925-2312. doi: https://doi.org/10.1016/j.neucom.2020.02.115.
  39. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
  40. iCaRL: Incremental Classifier and Representation Learning. In CVPR, 2017.
  41. Complementary learning for overcoming catastrophic forgetting using experience replay. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp.  3339–3345, 2019.
  42. Progressive Neural Networks, 2016. arXiv:1606.04671.
  43. Pseudo-rehearsal for continual learning with normalizing flows. In 4th Lifelong Machine Learning Workshop at ICML 2020, 2020.
  44. Generating high fidelity data from low-density regions using diffusion models. Computer Vision And Pattern Recognition, 2022. doi: 10.1109/CVPR52688.2022.01120.
  45. Continual Learning with Deep Generative Replay. In NeurIPS, 2017.
  46. Continual diffusion: Continual customization of text-to-image diffusion with c-lora. arXiv preprint arXiv:2304.06027, 2023.
  47. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp.  2256–2265. PMLR, 2015.
  48. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020.
  49. An empirical study of example forgetting during deep neural network learning. In International Conference on Learning Representations, 2018.
  50. Generative replay with feedback connections as a general strategy for continual learning, 2018. arXiv:1809.10635.
  51. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019.
  52. Brain-inspired replay for continual learning with artificial neural networks. Nature communications, 11(1):4069, 2020.
  53. Three types of incremental learning. Nature Machine Intelligence, 4(12):1185–1197, 2022.
  54. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018.
  55. Welling, M. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Learning, pp.  1121–1128, 2009.
  56. Memory replay gans: Learning to generate new categories without forgetting. In NeurIPS, 2018.
  57. Large scale incremental learning. In International Conference on Computer Vision, 2019.
  58. Lifelong Learning with Dynamically Expandable Networks. In ICLR, 2018.
  59. Iterative projection and matching: Finding structure-preserving representatives and its application to computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5414–5423, 2019.
  60. Exploring continual learning of diffusion models. arXiv preprint arXiv:2303.15342, 2023.
  61. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp.  3987–3995. PMLR, 2017.
  62. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pp.  12674–12685. PMLR, 2021.
  63. Dataset condensation with gradient matching. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=mSAKhLYLSsl.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Bartosz Cywiński (5 papers)
  2. Kamil Deja (27 papers)
  3. Tomasz Trzciński (116 papers)
  4. Bartłomiej Twardowski (37 papers)
  5. Łukasz Kuciński (20 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now