Feature Space Augmentation for Long-Tailed Data (2008.03673v1)

Abstract: Real-world data often follow a long-tailed distribution as the frequency of each class is typically different. For example, a dataset can have a large number of under-represented classes and a few classes with more than sufficient data. However, a model to represent the dataset is usually expected to have reasonably homogeneous performances across classes. Introducing class-balanced loss and advanced methods on data re-sampling and augmentation are among the best practices to alleviate the data imbalance problem. However, the other part of the problem about the under-represented classes will have to rely on additional knowledge to recover the missing information. In this work, we present a novel approach to address the long-tailed problem by augmenting the under-represented classes in the feature space with the features learned from the classes with ample samples. In particular, we decompose the features of each class into a class-generic component and a class-specific component using class activation maps. Novel samples of under-represented classes are then generated on the fly during training stages by fusing the class-specific features from the under-represented classes with the class-generic features from confusing classes. Our results on different datasets such as iNaturalist, ImageNet-LT, Places-LT and a long-tailed version of CIFAR have shown the state of the art performances.

• The paper introduces a method that decomposes class features using activation maps to generate synthetic samples for tail classes.
• It employs a two-phase training scheme, first learning base features and then fine-tuning with augmented samples, to improve decision boundaries.
• Experimental results on CIFAR, ImageNet-LT, and iNaturalist demonstrate a 3%-9% accuracy gain over existing imbalance mitigation techniques.

Feature Space Augmentation for Long-Tailed Data

The paper, "Feature Space Augmentation for Long-Tailed Data," presents a unique approach to address the challenges associated with training machine learning models on long-tailed datasets. These datasets, where a few classes have a disproportionately large number of samples while many others have significantly fewer, frequently hinder model performance due to class imbalance and insufficient data coverage. The technique introduced in this paper employs feature space augmentation to synthesize additional samples for under-represented classes, thereby improving their representation during training.

Methodological Overview

The proposed methodology decomposes features of each class into class-specific and class-generic components using class activation maps (CAMs). The class-generic features from classes with ample samples (head classes) are combined with class-specific features from the under-represented classes (tail classes) to generate new samples. This augmentation is performed in the feature space, as opposed to directly manipulating input data, which can introduce undesirable artifacts.

Two phases constitute the training scheme:

1. Initial Feature Learning (Phase-I): The entire dataset is used to train a feature extractor and a base classifier. This phase establishes foundational representations of the classes.
2. Feature Space Augmentation (Phase-II): During this phase, new samples are generated in the feature space for tail classes by mixing class-specific features from them with class-generic features sourced from confusing head classes. The augmented samples are then used to fine-tune the classifier.

This approach hypothesizes that head classes, due to their plethora of data, hold transferable knowledge in the form of class-generic features that can inform the under-represented tail classes. Theoretical underpinnings suggest that utilizing such information can help recover decision boundaries that are otherwise ill-defined due to data paucity in tail classes.

Experimental Results

The effectiveness of the proposed method is demonstrated through extensive experimentation on several datasets, including the CIFAR-10 and CIFAR-100 with artificially induced long-tailed properties, as well as the ImageNet-LT, Places-LT, and iNaturalist datasets.

• CIFAR Datasets: The proposed method significantly outperformed state-of-the-art techniques, including class-balanced loss and focal loss mechanisms, across various imbalance scenarios. Improvements ranged from 3% to 9% in classification accuracy over baseline methods on highly imbalanced datasets.
• ImageNet-LT and Places-LT: Evaluation on these large-scale datasets showed comparable or superior performance relative to other advanced techniques designed for long-tailed recognition, reinforcing the method's robustness.
• iNaturalist: The real-world applicability of the method is underscored by its performance on the iNaturalist datasets, often used to benchmark fine-grained and imbalanced class distributions. It consistently achieved higher accuracy compared to class-balanced approaches with conventional setups.

Implications and Future Directions

The approach effectively tailors data augmentation to the specifics of long-tailed distributions by leveraging the intrinsic properties of feature vectors in neural networks. Its capability to use class-generic features as a bridge to enhance the representation of tail classes in the feature space provides a promising route for dealing with imbalanced data scenarios.

Theoretically, the findings from this method support the hypothesis that linear separability and feature space dynamics can be instrumental in mitigating class imbalance issues. Practically, its end-to-end design allows integration with existing neural network architectures without significant modifications, offering ease of adoption in diverse applications.

For future research, extending this methodology to more complex data types beyond image recognition, such as natural language processing or graph-based tasks, could yield interesting insights. Additionally, exploring adaptive mechanisms to automatically identify class-generic versus class-specific features may enhance the flexibility and applicability of the approach, particularly in dynamically changing datasets or streaming data contexts.