Dynamic Few-Shot Visual Learning without Forgetting (1804.09458v1)

Abstract: The human visual system has the remarkably ability to be able to effortlessly learn novel concepts from only a few examples. Mimicking the same behavior on machine learning vision systems is an interesting and very challenging research problem with many practical advantages on real world vision applications. In this context, the goal of our work is to devise a few-shot visual learning system that during test time it will be able to efficiently learn novel categories from only a few training data while at the same time it will not forget the initial categories on which it was trained (here called base categories). To achieve that goal we propose (a) to extend an object recognition system with an attention based few-shot classification weight generator, and (b) to redesign the classifier of a ConvNet model as the cosine similarity function between feature representations and classification weight vectors. The latter, apart from unifying the recognition of both novel and base categories, it also leads to feature representations that generalize better on "unseen" categories. We extensively evaluate our approach on Mini-ImageNet where we manage to improve the prior state-of-the-art on few-shot recognition (i.e., we achieve 56.20% and 73.00% on the 1-shot and 5-shot settings respectively) while at the same time we do not sacrifice any accuracy on the base categories, which is a characteristic that most prior approaches lack. Finally, we apply our approach on the recently introduced few-shot benchmark of Bharath and Girshick [4] where we also achieve state-of-the-art results. The code and models of our paper will be published on: https://github.com/gidariss/FewShotWithoutForgetting

Dynamic Few-Shot Visual Learning without Forgetting

In their paper "Dynamic Few-Shot Visual Learning without Forgetting," Spyros Gidaris and Nikos Komodakis tackle the complex challenge of enabling visual recognition systems to learn new categories from minimal examples, while simultaneously retaining the ability to recognize previously learned categories. The inherent difficulty in this task lies in preventing catastrophic forgetting—where new information overwrites previously learned data.

The proposed solution consists of two primary innovations:

1. An attention-based few-shot classification weight generator.
2. A cosine-similarity-based ConvNet recognition model.

Few-Shot Classification Weight Generator

The few-shot classification weight generator plays a critical role in the dynamic learning process. During test time, this generator can create classification weight vectors for new categories using only a few examples (typically no more than five). The novelty of this approach lies in its use of attention mechanisms. These mechanisms allow the generator to focus selectively on the most relevant features from a set of base categories—those for which a substantial amount of training data is available.

Cosine-Similarity-Based ConvNet Recognition Model

The paper introduces a cosine-similarity-based classifier, which addresses the limitations of the traditional dot-product-based classifiers. The cosine similarity function normalizes both the feature vectors and classification weight vectors, ensuring that the magnitudes of these vectors do not affect the classification decisions. This normalization is particularly crucial when dealing with weight vectors for both base and novel categories, as the latter are dynamically generated and their magnitudes might otherwise diverge significantly.

The proposed system does away with the last ReLU activation in the network’s feature extractor to facilitate the negative values, enhancing the performance.

Numerical Results

The efficacy of the introduced system is demonstrated through extensive evaluations on the Mini-ImageNet dataset. The paper achieves notable results:

• 1-shot setting: Achieves 58.55% accuracy.
• 5-shot setting: Achieves 74.92% accuracy.

These results surpass prior state-of-the-art approaches, underlining the robustness of the proposed methods. The approach maintains the recognition accuracy for base categories around 70.88%-70.92%, showcasing its capacity for not forgetting previously learned information.

Implications and Future Developments

Practical Implications:

• Real-time interactive applications: The dynamic and computationally efficient few-shot learning method can be particularly beneficial for real-time applications on portable devices.
• Enhanced adaptability: This system can be employed in applications requiring frequent updates, such as security systems and content recommendation engines.

Theoretical Implications:

• Better generalization: The cosine similarity classifier inherently leads to feature representations that generalize better to unseen categories.
• Unified classification: The approach successfully unifies the recognition of base and novel categories, an achievement that had been elusive in previous research.

Future Research:

• Scalability: Further research could explore how to extend this system's scalability, particularly when dealing with a much larger number of categories.
• Adaptation to other domains: Testing the adaptability of this system in domains other than image classification (e.g., natural language processing) could be an intriguing direction.
• Hybrid architectures: Integrating this approach with other meta-learning or reinforcement learning paradigms might yield even more robust and adaptable AI systems.

In conclusion, the methods proposed by Gidaris and Komodakis offer substantial improvements in the field of few-shot learning by addressing and mitigating catastrophic forgetting. This paper presents significant advancements that promise meaningful applications in both academic research and practical deployment of machine learning systems.