Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GRASP: A Rehearsal Policy for Efficient Online Continual Learning (2308.13646v2)

Published 25 Aug 2023 in cs.LG, cs.CL, and cs.CV

Abstract: Continual learning (CL) in deep neural networks (DNNs) involves incrementally accumulating knowledge in a DNN from a growing data stream. A major challenge in CL is that non-stationary data streams cause catastrophic forgetting of previously learned abilities. A popular solution is rehearsal: storing past observations in a buffer and then sampling the buffer to update the DNN. Uniform sampling in a class-balanced manner is highly effective, and better sample selection policies have been elusive. Here, we propose a new sample selection policy called GRASP that selects the most prototypical (easy) samples first and then gradually selects less prototypical (harder) examples. GRASP has little additional compute or memory overhead compared to uniform selection, enabling it to scale to large datasets. Compared to 17 other rehearsal policies, GRASP achieves higher accuracy in CL experiments on ImageNet. Compared to uniform balanced sampling, GRASP achieves the same performance with 40% fewer updates. We also show that GRASP is effective for CL on five text classification datasets.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. Memory retention–the synaptic stability versus plasticity dilemma. Trends in neurosciences, 28(2):73–78, 2005.
  2. Rodeo: Replay for online object detection. In BMVC, 2020.
  3. Rapid adaptation in online continual learning: Are we evaluating it right? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  18852–18861, 2023.
  4. Online continual learning with maximally interfered retrieval. In NeurIPS, 2019a.
  5. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019b.
  6. Rainbow memory: Continual learning with a memory of diverse samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  8218–8227, 2021.
  7. Il2m: Class incremental learning with dual memory. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  583–592, 2019.
  8. Continual lifelong learning in natural language processing: A survey. In Proceedings of the 28th International Conference on Computational Linguistics, pp.  6523–6541, 2020.
  9. Nevis’22: A stream of 100 tasks sampled from 30 years of computer vision research. arXiv preprint arXiv:2211.11747, 2022.
  10. Coresets via bilevel optimization for continual learning and streaming. Advances in Neural Information Processing Systems, 33:14879–14890, 2020.
  11. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920–15930, 2020.
  12. Online learned continual compression with adaptive quantization modules. In International conference on machine learning, pp.  1240–1250. PMLR, 2020.
  13. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2021.
  14. Automated scalable bayesian inference via hilbert coresets. The Journal of Machine Learning Research, 20(1):551–588, 2019.
  15. Unsupervised learning of visual features by contrasting cluster assignments. arXiv preprint arXiv:2006.09882, 2020.
  16. End-to-end incremental learning. In Proceedings of the European conference on computer vision (ECCV), pp.  233–248, 2018.
  17. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pp.  532–547, 2018a.
  18. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018b.
  19. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486, 2019a.
  20. Continual learning with tiny episodic memories. arXiv preprint arXiv:1902.10486, 2019, 2019b.
  21. Bad students make great teachers: Active learning accelerates large-scale visual understanding. arXiv preprint arXiv:2312.05328, 2023.
  22. Deep bayesian active learning with image data. In International conference on machine learning, pp.  1183–1192. PMLR, 2017.
  23. Self-supervised training enhances online continual learning. In British Machine Vision Conference (BMVC), 2021.
  24. A bio-inspired incremental learning architecture for applied perceptual problems. Cognitive Computation, 8(5):924–934, 2016.
  25. Overcoming the stability gap in continual learning. arXiv preprint arXiv:2306.01904, 2023.
  26. How efficient are today’s continual learning algorithms? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp.  2430–2435, June 2023a.
  27. SIESTA: Efficient online continual learning with sleep. Transactions on Machine Learning Research, 2023b. ISSN 2835-8856. URL https://openreview.net/forum?id=MqDVlBWRRV.
  28. Selective replay enhances learning in online continual analogical reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  3502–3512, 2021.
  29. Online continual learning for embedded devices. In CoLLAs, 2022.
  30. New metrics and experimental paradigms for continual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp.  2031–2034, 2018.
  31. Memory efficient experience replay for streaming learning. In 2019 International Conference on Robotics and Automation (ICRA), pp.  9769–9776. IEEE, 2019.
  32. Remind your neural network to prevent catastrophic forgetting. In European Conference on Computer Vision, pp.  466–483. Springer, 2020.
  33. Replay in deep learning: Current approaches and missing biological elements. Neural Computation, 33(11):2908–2950, 2021.
  34. Exemplar-supported generative reproduction for class incremental learning. In BMVC, pp.  98, 2018.
  35. Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  1314–1324, 2019.
  36. Overcoming catastrophic forgetting for continual learning via model adaptation. In International conference on learning representations, 2018.
  37. Continual learning for text classification with information disentanglement based regularization. arXiv preprint arXiv:2104.05489, 2021.
  38. Memory-efficient incremental learning through feature adaptation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pp.  699–715. Springer, 2020.
  39. Ordered sgd: A new stochastic optimization framework for empirical risk minimization. In International Conference on Artificial Intelligence and Statistics, pp.  669–679. PMLR, 2020.
  40. Continual learning of natural language processing tasks: A survey. arXiv preprint arXiv:2211.12701, 2022.
  41. Fearnet: Brain-inspired model for incremental learning. In ICLR, 2018.
  42. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
  43. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pp.  4171–4186, 2019.
  44. Online continual learning on class incremental blurry task configuration with anytime inference. In International Conference on Learning Representations, 2021.
  45. Generative feature replay for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp.  226–227, 2020.
  46. Large-scale long-tailed recognition in an open world. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  2537–2546, 2019.
  47. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
  48. Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. In International Conference on Learning Representations, 2021.
  49. Learning to remember: A synaptic plasticity driven framework for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  11321–11329, 2019.
  50. Play it again: reactivation of waking experience and memory. Trends in neurosciences, 33(5):220–229, 2010.
  51. Latent replay for real-time continual learning. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.  10203–10209. IEEE, 2020.
  52. Gdumb: A simple approach that questions our progress in continual learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pp.  524–540. Springer, 2020.
  53. Computationally budgeted continual learning: What does matter? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  3698–3707, 2023a.
  54. Online continual learning without the storage constraint. arXiv preprint arXiv:2305.09253, 2023b.
  55. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp.  2001–2010, 2017.
  56. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations. International Conference on Learning Representations, ICLR, 2019.
  57. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015.
  58. Active hidden markov models for information extraction. In International symposium on intelligent data analysis, pp.  309–318. Springer, 2001.
  59. Online class-incremental continual learning with adversarial shapley value. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.  9630–9638, 2021.
  60. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
  61. Super-convergence: Very fast training of neural networks using large learning rates. arxiv. arXiv preprint arXiv:1708.07120, 2017.
  62. Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35:19523–19536, 2022.
  63. Topology-preserving class-incremental learning. In European Conference on Computer Vision, pp.  254–270. Springer, 2020.
  64. Three types of incremental learning. Nature Machine Intelligence, pp.  1–13, 2022.
  65. Continual learning: Applications and the road forward. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=axBIMcGZn9.
  66. Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37–57, 1985.
  67. A comprehensive survey of continual learning: Theory, method and application. arXiv preprint arXiv:2302.00487, 2023.
  68. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems, 4:795–813, 2022.
  69. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  374–382, 2019.
  70. Incremental learning using conditional adversarial networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.  6619–6628, 2019.
  71. Online coreset selection for rehearsal-based continual learning. In International Conference on Learning Representations, 2022.
  72. Memory-efficient class-incremental learning for image classification. IEEE Transactions on Neural Networks and Learning Systems, 33(10):5966–5977, 2021.
  73. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017.
  74. Deep class-incremental learning: A survey. arXiv preprint arXiv:2302.03648, 2023.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube