DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification (1809.00981v1)

Abstract: Deep learning has revolutionized the performance of classification, but meanwhile demands sufficient labeled data for training. Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else. Such regimes arise from practical situations where not only data labeling but also data collection itself is expensive. We propose a deep adversarial data augmentation (DADA) technique to address the problem, in which we elaborately formulate data augmentation as a problem of training a class-conditional and supervised generative adversarial network (GAN). Specifically, a new discriminator loss is proposed to fit the goal of data augmentation, through which both real and augmented samples are enforced to contribute to and be consistent in finding the decision boundaries. Tailored training techniques are developed accordingly. To quantitatively validate its effectiveness, we first perform extensive simulations to show that DADA substantially outperforms both traditional data augmentation and a few GAN-based options. We then extend experiments to three real-world small labeled datasets where existing data augmentation and/or transfer learning strategies are either less effective or infeasible. All results endorse the superior capability of DADA in enhancing the generalization ability of deep networks trained in practical extremely low data regimes. Source code is available at https://github.com/SchafferZhang/DADA.

DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification

The paper presents a novel approach to addressing the challenges of training deep learning models in extremely low data regimes through the introduction of Deep Adversarial Data Augmentation (DADA). This work leverages the powerful generative capabilities of GANs to create augmented data that aids in training classifiers when only limited labeled data is available and when it is practically infeasible to attain additional unlabeled data for semi-supervised learning.

Key Contributions

1. Learning-Based Data Augmentation: Unlike traditional patient-specific or empirical augmentation strategies, DADA is a class-conditional learning-based approach. It uses a GAN-like framework where the generator creates augmented samples conditioned on class labels, and the discriminator functions as a classifier ensuring both real and generated samples contribute to refining decision boundaries.
2. Novel Discriminator Loss Function: The paper proposes a 2$k$ loss function that optimizes the discriminator output over augmented samples. This robust class-conditional treatment enables the generated samples to contribute effectively to the classifier's learning process, aligning decision boundaries between real and synthetic data. The proposed 2$k$ loss provides improved outcomes compared to the traditional $k+1$ loss in semi-supervised GANs.
3. Validation and Generalization Performance: Through extensive experiments across benchmark datasets (CIFAR-10, CIFAR-100, SVHN) and real-world datasets (KDEF, BCI Competition EEG data, CBIS-DDSM), DADA consistently enhances classifier performance in extremely low data conditions. The results demonstrate DADA’s capability in boosting generalization and achieving competitive accuracy compared to traditional data augmentation techniques and transfer learning approaches.

Experimental Findings

The validation on benchmark datasets reveals significant improvement in classifier accuracy with DADA, especially when available samples per class are scarce (e.g., less than 400 images per class in CIFAR-10). The method proves particularly beneficial in cases where traditional augmentation strategies fall short or when drawing sophisticated boundaries in data distributions, such as with the SVHN dataset where outliers can frequently occur.

In real-world applications like EEG signal classification, DADA shows promise by surpassing existing classification methods. It notably succeeds in augmenting data without reliance on domain-specific pre-processing, which typically hampers other data synthesis approaches. DADA’s data-driven augmentation approach proves advantageous for medical imaging and emotion recognition tasks as well, where data collection and labeling are inherently expensive and challenging, attesting to the adaptability and application breadth of the proposed method.

Implications for AI Development

DADA exemplifies an innovative stride in exploiting generative models for classification tasks beyond conventional training paradigms, extending the utility of GANs into a fully-supervised field. This has implications for developing advanced AI systems that can operate efficiently with limited data resources. The method may further impact future research in areas where data scarcity is a critical issue, potentially inspiring advancements in few-shot learning and generalized data synthesis techniques.

Future Directions

Considering the promising results and adaptability of DADA, future studies could explore improvements in generator models to further enhance the semantic diversity of augmented samples. Additionally, integrating DADA into more complex deep learning architectures may yield new opportunities for reinforcing learning capabilities in restrictive data environments. Cross-disciplinary applications, particularly within healthcare and defense, might heavily benefit from such methodologies, catalyzing the adoption of AI solutions in data-constrained domains.