Dense Classification and Implanting for Few-Shot Learning (1903.05050v1)

Abstract: Training deep neural networks from few examples is a highly challenging and key problem for many computer vision tasks. In this context, we are targeting knowledge transfer from a set with abundant data to other sets with few available examples. We propose two simple and effective solutions: (i) dense classification over feature maps, which for the first time studies local activations in the domain of few-shot learning, and (ii) implanting, that is, attaching new neurons to a previously trained network to learn new, task-specific features. On miniImageNet, we improve the prior state-of-the-art on few-shot classification, i.e., we achieve 62.5%, 79.8% and 83.8% on 5-way 1-shot, 5-shot and 10-shot settings respectively.

• The paper proposes two novel methods, dense classification and neural implanting, to enhance generalization and adaptability in few-shot learning for computer vision.
• Dense classification improves accuracy by performing classification over raw feature maps at all spatial locations, leveraging spatial information more effectively than pooled features.
• Neural implanting adds new, task-specific neurons to top network layers, allowing rapid adaptation to new tasks without forgetting previously learned features.

Dense Classification and Implanting for Few-Shot Learning: A Summary

The paper "Dense Classification and Implanting for Few-Shot Learning" by Lifchitz et al. focuses on advancing few-shot learning methodologies in computer vision by proposing two novel approaches: dense classification over feature maps and neural implanting. This summary explicates the core contributions and results of the paper and discusses their significance within the field.

Context and Challenges

Few-shot learning (FSL) addresses the challenge of generalizing deep learning models trained on a limited number of samples, which is crucial in domains where acquiring large amounts of labeled data is impractical. Traditionally, the successful application of deep neural networks (DNNs) relies on vast datasets. However, their effectiveness in low-data regimes is compromised due to overfitting and lack of generalization, often encountered in real-world applications.

Contributions

The paper presents two principal contributions to tackle these challenges:

1. Dense Classification: Contrary to common practices where feature maps are pooled into vectors before classification, this approach maintains spatial structure by performing classification over raw feature maps. This method leverages localized supervision and encourages correct predictions at all pixel locations, thereby enhancing the network's ability to utilize spatial information. Empirical results show that this leads to smoother class activation maps and heightened accuracy in novel class tasks on benchmarks like miniImageNet.
2. Neural Implanting: Borrowing concepts from incremental and transfer learning, the authors introduce neural implants, which involve adding new, task-specific neurons to an existing network. This mechanism allows for rapid adaptation to new tasks without erasing previously learned features, a common issue in fine-tuning approaches. The implants are selectively applied to the network's top layers, which bear more task-specific features, and the rest of the network remains fixed to prevent overfitting and to preserve previously acquired knowledge.

Method and Architecture

The proposed architecture includes using a ResNet-12 backbone for representations, which has shown significant performance improvements over shallower networks such as C128F. Dense classification modifies the learning objective by applying the classification loss to each spatial location within the feature maps. Implanting involves adding convolutive layers at the higher levels of the ResNet, which trains only these newly added parameters for new tasks.

Experimental Results

The proposed methods were evaluated against standard datasets, miniImageNet and FC100. The comprehensive experimentation demonstrated:

• Dense classification consistently improves the classification accuracy on novel classes.
• The introduction of neural implants results in further accuracy gains, surpassing the previous state-of-the-art, particularly noticeable in settings involving more support examples.
• The experiments also revealed the advantage of deeper networks like ResNet-12, where dense classification exhibits greater benefit due to their higher receptive field.

Implications and Future Directions

The implications of these findings extend the versatility of few-shot learning by providing a framework that reconciles base knowledge retention with adaptability, a balance crucial for the successful deployment of AI models in dynamic environments. This work could inspire future studies exploring more refined strategies for neural implanting and spatial feature utilization across other low-data learning paradigms.

In conclusion, by addressing few-shot learning challenges through dense classification and neural implanting, the paper presents significant advancements that enhance generalization and adaption, paving the way for future explorations and practical AI applications in data-scarce contexts.