Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Class-wise Classifier Design Capable of Continual Learning using Adaptive Resonance Theory-based Topological Clustering (2203.09879v2)

Published 18 Mar 2022 in cs.LG

Abstract: This paper proposes a supervised classification algorithm capable of continual learning by utilizing an Adaptive Resonance Theory (ART)-based growing self-organizing clustering algorithm. The ART-based clustering algorithm is theoretically capable of continual learning, and the proposed algorithm independently applies it to each class of training data for generating classifiers. Whenever an additional training data set from a new class is given, a new ART-based clustering will be defined in a different learning space. Thanks to the above-mentioned features, the proposed algorithm realizes continual learning capability. Simulation experiments showed that the proposed algorithm has superior classification performance compared with state-of-the-art clustering-based classification algorithms capable of continual learning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation 1989, 24, 109–165.
  2. Continual lifelong learning with neural networks: A review. Neural Networks 2019, 113, 57–71.
  3. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734 2019.
  4. Localizing catastrophic forgetting in neural networks. arXiv preprint arXiv:1906.02568 2019.
  5. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 2017, 114, 3521–3526.
  6. An incremental network for on-line unsupervised classification and topology learning. Neural Networks 2006, 19, 90–106.
  7. Fritzke, B. A growing neural gas network learns topologies. Advances in Neural Information Processing Systems 1995, 7, 625–632.
  8. A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Networks 2008, 21, 1537–1547.
  9. Lifelong learning of human actions with deep neural network self-organization. Neural Networks 2017, 96, 137–149.
  10. A self-organizing incremental neural network for continual supervised learning. Expert Systems with Applications 2021, 185, 115662.
  11. Grossberg, S. Competitive learning: From interactive activation to adaptive resonance. Cognitive Science 1987, 11, 23–63.
  12. Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Transactions on Signal Processing 2007, 55, 5286–5298.
  13. Self-organizing maps with information theoretic learning. Neurocomputing 2015, 147, 3–14.
  14. Topological Clustering via Adaptive Resonance Theory With Information Theoretic Learning. IEEE Access 2019, 7, 76920–76936.
  15. Fast Topological Adaptive Resonance Theory Based on Correntropy Induced Metric. In Proceedings of the Proceedings of IEEE Symposium Series on Computational Intelligence, 2019, pp. 2215–2221.
  16. Adaptive resonance theory-based topological clustering with a divisive hierarchical structure capable of continual learning. arXiv preprint arXiv:2201.10713 2022.
  17. Finite mixture models. Annual Review of Statistics and its Application 2019, 6, 355–378.
  18. Lloyd, S. Least squares quantization in PCM. IEEE Transactions on Information Theory 1982, 28, 129–137.
  19. The ART of adaptive pattern recognition by a self-organizing neural network. Computer 1988, 21, 77–88.
  20. SOINN+, a self-organizing incremental neural network for unsupervised learning from noisy data streams. Expert Systems with Applications 2020, 143, 113069.
  21. A self-organising network that grows when required. Neural Networks 2002, 15, 1041–1058.
  22. Evolutionary fuzzy ARTMAP neural networks for classification of semiconductor defects. IEEE Transactions on Neural Networks and Learning Systems 2014, 26, 933–950.
  23. OnARTMAP: A fuzzy ARTMAP-based architecture. Neural Networks 2018, 98, 236–250.
  24. A novel fuzzy ARTMAP with area of influence. Neurocomputing 2021, 432, 80–90.
  25. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Networks 1991, 4, 759–771.
  26. The Bayesian ARTMAP. IEEE Transactions on Neural Networks 2007, 18, 1628–1644.
  27. Incremental Local Distribution-Based Clustering Using Bayesian Adaptive Resonance Theory. IEEE Transactions on Neural Networks and Learning Systems 2019, 30, 3496–3504.
  28. Distributed dual vigilance fuzzy adaptive resonance theory learns online, retrieves arbitrarily-shaped clusters, and mitigates order dependence. Neural Networks 2020, 121, 208–228.
  29. Kernel Bayesian ART and ARTMAP. Neural Networks 2018, 98, 76–86.
  30. A Kernel Bayesian Adaptive Resonance Theory with a Topological Structure. International Journal of Neural Systems 2019, 29, 1850052 (20 pages).
  31. Dual vigilance fuzzy adaptive resonance theory. Neural Networks 2019, 109, 1–5.
  32. A comprehensive study of class incremental learning algorithms for visual tasks. Neural Networks 2021, 135, 38–54.
  33. Continual learning through synaptic intelligence. In Proceedings of the Proceedings of International Conference on Machine Learning, 2017, pp. 3987–3995.
  34. Continual learning with deep generative replay. In Proceedings of the Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, pp. 2994–3003.
  35. Variational continual learning. In Proceedings of the Proceedings of International Conference on Learning Representations, 2018, pp. 1–18.
  36. An open-ended continual learning for food recognition using class incremental extreme learning machines. IEEE Access 2020, 8, 82328–82346.
  37. An incremental kernel extreme learning machine for multi-label learning with emerging new labels. IEEE Access 2020, 8, 46055–46070.
  38. Lifelong learning of spatiotemporal representations with dual-memory recurrent self-organization. Frontiers in Neurorobotics 2018, 12, 78.
  39. Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 1992, 3, 698–713.
  40. Normal reference bandwidths for the general order, multivariate kernel density derivative estimator. Statistics & Probability Letters 2012, 82, 2198–2205.
  41. Silverman, B.W. Density estimation for statistics and data analysis; Routledge, 2018.
  42. Multi-label classification based on adaptive resonance theory. In Proceedings of the Proceedins of 2020 IEEE Symposium Series on Computational Intelligence, 2020, pp. 1913–1920.
  43. K-means properties on six clustering benchmark datasets. Applied Intelligence 2018, 48, 4743–4759.
  44. A survey of deep neural network architectures and their applications. Neurocomputing 2017, 234, 11–26.
  45. UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, 2019.
  46. Cluster ensembles—A knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research 2002, 3, 583–617.
  47. Comparing partitions. Journal of Classification 1985, 2, 193–218.
  48. Demšar, J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 2006, 7, 1–30.
  49. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering 2018, 31, 2346–2363.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now