Generating Classification Weights with GNN Denoising Autoencoders for Few-Shot Learning (1905.01102v1)

Abstract: Given an initial recognition model already trained on a set of base classes, the goal of this work is to develop a meta-model for few-shot learning. The meta-model, given as input some novel classes with few training examples per class, must properly adapt the existing recognition model into a new model that can correctly classify in a unified way both the novel and the base classes. To accomplish this goal it must learn to output the appropriate classification weight vectors for those two types of classes. To build our meta-model we make use of two main innovations: we propose the use of a Denoising Autoencoder network (DAE) that (during training) takes as input a set of classification weights corrupted with Gaussian noise and learns to reconstruct the target-discriminative classification weights. In this case, the injected noise on the classification weights serves the role of regularizing the weight generating meta-model. Furthermore, in order to capture the co-dependencies between different classes in a given task instance of our meta-model, we propose to implement the DAE model as a Graph Neural Network (GNN). In order to verify the efficacy of our approach, we extensively evaluate it on ImageNet based few-shot benchmarks and we report strong results that surpass prior approaches. The code and models of our paper will be published on: https://github.com/gidariss/wDAE_GNN_FewShot

Generating Classification Weights with GNN Denoising Autoencoders for Few-Shot Learning

The paper by Spyros Gidaris and Nikos Komodakis presents an innovative approach to advancing few-shot learning through the development of a novel meta-model that adapts existing recognition models to novel classes with limited training examples. The proposed method leverages Graph Neural Networks (GNNs) and Denoising Autoencoders (DAEs) to generate classification weight vectors for both base and novel classes, achieving superior performance in few-shot settings.

Key Contributions

The primary contribution of this work lies in its unique combination of denoising autoencoders and graph neural networks. The introduced methodology uses a DAE that denoises classification weights by reconstructing them from Gaussian-noise-corrupted versions. This component provides regularization, mitigating overfitting issues prevalent in few-shot learning models. The paper achieves a secondary critical contribution by designing this DAE as a GNN, which allows capturing the intricate dependencies between classes in the feature space, leading to more robust classification.

Methodology

The proposed framework involves a two-phasic operation. Initially, for a recognition model trained on a set of base classes, novel class recognition is facilitated by estimating novel classification weight vectors through data-driven methods. Subsequently, a DAE model, built as a GNN, refinements these weight vectors by learning and exploiting the structural dependencies among them. Such structural understanding allows the model to predict classification weights more effectively by considering inter-class relationships typically found within task instances.

Experimental Comparisons and Results

The researchers evaluate their method on several benchmarks: ImageNet-FS, MiniImageNet, and tiered-MiniImageNet. The framework outperformed existing few-shot learning methods, particularly in challenging training settings with fewer samples per class (K = 1 or 5). Notably, significant improvements are recorded in top-5 accuracy metrics on ImageNet-FS, as well as top-1 accuracies on MiniImageNet and tiered-MiniImageNet, underscoring the effectiveness of incorporating DAE under the GNN architecture.

Theoretical and Practical Implications

From a theoretical standpoint, the paper demonstrates the potential of integrating DAEs and GNNs to harness both noise correction and relational inference. Practically, this offers a reliable method for evolving existing models to recognize new classes with minimal data, a typical constraint in real-world applications like medical image diagnostics and rare species identification.

Future Research Directions

Looking forward, this work might inspire extensions into various other domains of few-shot learning. For instance, its application could be broadened beyond image classification to other domains where class dependencies play a vital role. Research might also explore optimizing the balance between noise regularization and learning capacity, potentially enhancing the adaptability of models trained under such paradigms.

In conclusion, the fusion of DAEs and GNNs in generating classification weights for few-shot learning creates promising avenues for both academic research and practical deployability, as this method markedly improves the agility of recognition models in adapting to novel class recognition tasks. The insights provided by this work could catalyze further advances in the meta-learning landscape, driving refined methodologies in model parameter generation.