Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Lifelong Continual Learning for Anomaly Detection: New Challenges, Perspectives, and Insights (2303.07557v2)

Published 14 Mar 2023 in cs.LG and cs.AI

Abstract: Anomaly detection is of paramount importance in many real-world domains, characterized by evolving behavior. Lifelong learning represents an emerging trend, answering the need for machine learning models that continuously adapt to new challenges in dynamic environments while retaining past knowledge. However, limited efforts are dedicated to building foundations for lifelong anomaly detection, which provides intrinsically different challenges compared to the more widely explored classification setting. In this paper, we face this issue by exploring, motivating, and discussing lifelong anomaly detection, trying to build foundations for its wider adoption. First, we explain why lifelong anomaly detection is relevant, defining challenges and opportunities to design anomaly detection methods that deal with lifelong learning complexities. Second, we characterize learning settings and a scenario generation procedure that enables researchers to experiment with lifelong anomaly detection using existing datasets. Third, we perform experiments with popular anomaly detection methods on proposed lifelong scenarios, emphasizing the gap in performance that could be gained with the adoption of lifelong learning. Overall, we conclude that the adoption of lifelong anomaly detection is important to design more robust models that provide a comprehensive view of the environment, as well as simultaneous adaptation and knowledge retention.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Charu C Aggarwal. An introduction to outlier analysis. In Outlier analysis, pages 1–34. Springer, 2017.
  2. Anomaly detection in time series: a comprehensive evaluation. Proceedings of the VLDB Endowment, 15(9):1779–1797, 2022.
  3. Autoencoder-based ids for cloud and mobile devices. In 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pages 728–736. IEEE, 2021.
  4. Spatially-aware autoencoders for detecting contextual anomalies in geo-distributed data. In International Conference on Discovery Science, pages 461–471. Springer, 2021.
  5. Anomaly detection, analysis and prediction techniques in iot environment: A systematic literature review. IEEE Access, 7:81664–81681, 2019.
  6. Using an autoencoder in the design of an anomaly detector for smart manufacturing. Pattern Recognition Letters, 136:272–278, 2020.
  7. Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR), 54(2):1–38, 2021.
  8. Deep learning for anomaly detection: A survey. arXiv preprint arXiv:1901.03407, 2019.
  9. Deep learning for anomaly detection: A review. ACM Comput. Surv., 54(2), mar 2021.
  10. Drift-aware methodology for anomaly detection in smart grid. IEEE Access, 7:9645–9657, 2019.
  11. Anomaly detection of calcifications in mammography based on 11,000 negative cases. IEEE Transactions on Biomedical Engineering, 69(5):1639–1650, 2022.
  12. Scalable auto-encoders for gravitational waves detection from time series data. Expert Systems with Applications, 151:113378, 2020.
  13. Online learning and sequential anomaly detection in trajectories. IEEE transactions on pattern analysis and machine intelligence, 36(6):1158–1173, 2013.
  14. Learning deep event models for crowd anomaly detection. Neurocomputing, 219:548–556, 2017.
  15. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54 – 71, 2019.
  16. Continual learning for recurrent neural networks: an empirical evaluation. Neural Networks, 143:607–627, 2021.
  17. Joao Gama. Knowledge discovery from data streams. CRC Press, 2010.
  18. On the role of neurogenesis in overcoming catastrophic forgetting. arXiv preprint arXiv:1811.02113, 2018.
  19. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
  20. Continual learning: A review of techniques, challenges and future directions. IEEE Transactions on Artificial Intelligence, 1(01):1–21, dec 5555.
  21. Overcoming catastrophic forgetting in graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8653–8661, 2021.
  22. Using hindsight to anchor past knowledge in continual learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 6993–7001, 2021.
  23. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5513–5533, 2022.
  24. Policy and value transfer in lifelong reinforcement learning. In International Conference on Machine Learning, pages 20–29, 2018.
  25. Open-world machine learning: applications, challenges, and opportunities. ACM Computing Surveys, 55(10):1–37, 2023.
  26. A domain-agnostic approach for characterization of lifelong learning systems. Neural Networks, 160:274–296, 2023.
  27. A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning. Neural Networks, 160:306–336, 2023.
  28. Online active continual learning for robotic lifelong object recognition. IEEE Transactions on Neural Networks and Learning Systems, pages 1–15, 2023.
  29. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28–51, 2022.
  30. Distributed continual intrusion detection: A collaborative replay framework. In 2023 IEEE International Conference on Big Data (BigData), pages 3255–3263, Los Alamitos, CA, USA, dec 2023. IEEE Computer Society.
  31. Fuzzy logic-based ddos attacks and network traffic anomaly detection methods: Classification, overview, and future perspectives. Information Sciences, 2023.
  32. Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019.
  33. A comprehensive study of class incremental learning algorithms for visual tasks. Neural Networks, 135:38–54, 2021.
  34. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021.
  35. Adaptive neural networks for online domain incremental continual learning. In Poncelet Pascal and Dino Ienco, editors, Discovery Science, pages 89–103, Cham, 2022. Springer Nature Switzerland.
  36. Continual prototype evolution: Learning online from non-stationary data streams. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8250–8259, October 2021.
  37. A comprehensive empirical evaluation on online continual learning. arXiv preprint arXiv:2308.10328, 2023.
  38. Rehearsal-free continual learning over small non-i.i.d. batches. 1st Workshop on Continual Learning in Computer Vision at CVPR2020, 2019.
  39. Is class-incremental enough for continual learning? Frontiers in Artificial Intelligence, 5, 2022.
  40. Single-head lifelong learning based on distilling knowledge. IEEE Access, 10:35469–35478, 2022.
  41. Cnn features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 806–813, 2014.
  42. Core50: a new dataset and benchmark for continuous object recognition. In Conference on Robot Learning, pages 17–26. PMLR, 2017.
  43. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935–2947, 2018.
  44. Continual learning in practice. NeurIPS Continual Learning Workshop, 2018.
  45. Packnet: Adding multiple tasks to a single network by iterative pruning. arXiv, https://arxiv.org/abs/1711.05769, 2017.
  46. Forget-free continual learning with winning subnetworks. ICML, x, 2022.
  47. Rethinking experience replay: a bag of tricks for continual learning. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 2180–2187. IEEE, 2021.
  48. Continual learning with deep generative replay. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, NeurIPS, volume 30. Curran Associates, Inc., 2017.
  49. Heart: Heterogeneous log anomaly detection using robust transformers. In International Conference on Discovery Science, pages 673–687. Springer, 2023.
  50. Dataset for anomalies detection in 3d printing. In International Conference on Computational Science, pages 647–653. Springer, 2021.
  51. A comparative evaluation of unsupervised anomaly detection algorithms for multivariate data. PloS one, 11(4):e0152173, 2016.
  52. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  53. Support vector method for novelty detection. In Advances in neural information processing systems, pages 582–588, 2000.
  54. Lof: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000.
  55. Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining, pages 413–422. IEEE, 2008.
  56. Copod: copula-based outlier detection. In 2020 IEEE International Conference on Data Mining (ICDM), pages 1118–1123. IEEE, 2020.
  57. Arcade: A rapid continual anomaly detector. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 10449–10456. IEEE, 2021.
  58. Continual learning for anomaly detection in surveillance videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 254–255, 2020.
  59. Cpdga: Change point driven growing auto-encoder for lifelong anomaly detection. Knowledge-Based Systems, page 108756, 2022.
  60. Vlad: Task-agnostic vae-based lifelong anomaly detection. Neural Networks, 165:248–273, 2023.
  61. Active lifelong anomaly detection with experience replay. In The 9th IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2022.
  62. Lifelong anomaly detection through unlearning. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS ’19, page 1283–1297, New York, NY, USA, 2019. Association for Computing Machinery.
  63. One-class classifiers with incremental learning and forgetting for data streams with concept drift. Soft Computing, 19(12):3387–3400, 2015.
  64. Biological underpinnings for lifelong learning machines. Nature Machine Intelligence, 4(3):196–210, 2022.
  65. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018.
  66. Learning over subconcepts: Strategies for 1-class classification. Computational Intelligence, 34(2):440–467, 2018.
  67. Real-time evaluation in online continual learning: A new hope. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11888–11897, 2023.
  68. A new approach for developing segmentation algorithms for strongly imbalanced data. IEEE Access, 7:82970–82977, 2019.
  69. The class imbalance problem in deep learning. Machine Learning, pages 1–57, 2022.
  70. Don’t forget, there is more than forgetting: new metrics for continual learning, 2018.
  71. A detailed analysis of the kdd cup 99 data set. In 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, pages 1–6, 2009.
  72. Unsw-nb15: a comprehensive data set for network intrusion detection systems (unsw-nb15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS), pages 1–6, 2015.
  73. Anomaly detection and repair for accurate predictions in geo-distributed big data. Big Data Research, 16:18–35, 2019.
  74. A higher prediction accuracy–based alpha–beta filter algorithm using the feedforward artificial neural network. CAAI Transactions on Intelligence Technology, 2022.
  75. An improved alpha beta filter using a deep extreme learning machine. IEEE Access, 9:61548–61564, 2021.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now