Multi-View Class Incremental Learning (2306.09675v3)

Abstract: Multi-view learning (MVL) has gained great success in integrating information from multiple perspectives of a dataset to improve downstream task performance. To make MVL methods more practical in an open-ended environment, this paper investigates a novel paradigm called multi-view class incremental learning (MVCIL), where a single model incrementally classifies new classes from a continual stream of views, requiring no access to earlier views of data. However, MVCIL is challenged by the catastrophic forgetting of old information and the interference with learning new concepts. To address this, we first develop a randomization-based representation learning technique serving for feature extraction to guarantee their separate view-optimal working states, during which multiple views belonging to a class are presented sequentially; Then, we integrate them one by one in the orthogonality fusion subspace spanned by the extracted features; Finally, we introduce selective weight consolidation for learning-without-forgetting decision-making while encountering new classes. Extensive experiments on synthetic and real-world datasets validate the effectiveness of our approach.

• The paper introduces MVCNet, a novel framework that integrates multi-view features with incremental learning to prevent catastrophic forgetting.
• It employs a randomization-based representation and orthogonality fusion strategy, ensuring effective adaptation without relying on past data.
• MVCNet demonstrates superior generalization in dynamic environments, paving the way for real-world AI applications in robotics and complex data systems.

The MVCIL Paradigm

Multi-view learning (MVL) has been an area of interest due to its capacity to leverage varied perspectives within datasets, thereby enhancing the performance of AI systems. The research presents an intriguing development towards a more practical application of MVL called multi-view class incremental learning (MVCIL). MVCIL is designed to cope with the challenges posed by dynamic environments where traditional MVL models fall short. Specifically, this entails integrating progressive data observations from multiple views to learn new classes without forgetting previously acquired information.

Addressing Catastrophic Forgetting

Central to the paper is the investigation of catastrophic forgetting—a major hurdle in developing systems that must adapt to new information without losing prior knowledge. The authors propose an innovative model, dubbed MVCNet, which continually updates a single model to classify incrementally presented new classes. A significant component of MVCNet is a randomization-based representation learning strategy, which ensures that multiple views associated with a class maintain their view-optimized functioning states. Additionally, an orthogonality fusion concept has been introduced, optimizing the integration of multiple views into a common subspace that maintains the integrity of past-view information.

MVCNet: Architecture and Performance

The MVCNet architecture follows a well-defined three-phase pipeline: feature extraction through random mapping and representation learning, orthogonality fusion to integrate new views without requiring past data, and selective weight consolidation to mitigate catastrophic forgetting. The numerical results are compelling, demonstrating MVCNet's effectiveness in retaining knowledge and integrating new concepts. The authors underline MVCNet's strength in generalization, illustrating its superior robustness when confronted with familiar but previously untrained views, in comparison to other methods.

Implications and Future Research

The paper serves as an extended line of inquiry from existing MVL approaches, offering a fresh direction for future research. One of its core contributions is the embodiment of MVCNet, a methodological innovation capable of handling continual views without the dependence on a fixed dataset nor the need for massive computational resources. This opens up possibilities for MVCNet's application in real-world scenarios, such as robotics and complex data systems where new classes of data can emerge unpredictably.

Furthermore, the authors suggest expanding the MVCIL paradigm to encompass scenarios involving incomplete views or trusted views. This points toward the necessity of creating AI systems that can learn and update their knowledge base in real-time, which is crucial in continually evolving digital landscapes.

Given the proposed MVCNet's encouraging results, future scholarly attention can be expected in refining and extending these models to handle the nuances involved with an unlimited number of views and classes. As AI systems are progressively introduced into dynamic real-world environments, strategies such as MVCNet will become more significant, highlighting the need to create models that not only learn effectively but can also do so continuously over time.