Masking: A New Perspective of Noisy Supervision (1805.08193v2)

Abstract: It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. However, such estimation is practically difficult, due to either the indirect nature of two-step approaches, or not big enough data to afford end-to-end approaches. In this paper, we propose a human-assisted approach called Masking that conveys human cognition of invalid class transitions and naturally speculates the structure of the noise transition matrix. To this end, we derive a structure-aware probabilistic model incorporating a structure prior, and solve the challenges from structure extraction and structure alignment. Thanks to Masking, we only estimate unmasked noise transition probabilities and the burden of estimation is tremendously reduced. We conduct extensive experiments on CIFAR-10 and CIFAR-100 with three noise structures as well as the industrial-level Clothing1M with agnostic noise structure, and the results show that Masking can improve the robustness of classifiers significantly.

• The paper proposes a structure-aware probabilistic model that leverages human insights to simplify noise transition matrix estimation.
• It introduces a tempered sigmoid function and GAN-based alignment to effectively capture and utilize cognitive biases in data.
• Empirical results on CIFAR-10, CIFAR-100, and Clothing1M show significant accuracy gains compared to traditional noisy supervision methods.

Exploring a Novel Approach to Noisy Supervision in Classifier Training

The paper "Masking: A New Perspective of Noisy Supervision" introduces an innovative methodology designed to train classifiers effectively in the presence of noisy ground-truth labels, a common issue in various data collection scenarios. The conventional challenge addressed is the difficulty in accurately estimating a noise transition matrix, which describes how true labels transition into noisy ones. The standard approaches are either indirect or data-intensive, thus limiting their practical applicability.

Summary of Contributions

The paper proposes a novel technique termed "Masking," which leverages human cognition to identify and exclude invalid class transitions to infer the structure of the noise transition matrix. The Masking method introduces a probabilistic model that integrates a structure prior, overcoming the challenges in structure extraction and alignment. The paper offers the following key contributions:

1. Structure-Aware Probabilistic Model: Incorporates human-inferred structures in estimating the noise transition matrix. By identifying valid and invalid transitions, the model reduces the complexity of the estimation task.
2. Tempered Sigmoid Function: Introduced to capture human cognitive biases in structure extraction, facilitating the alignment with the observed data.
3. GAN-Based Structure Alignment: Employs a variant of Generative Adversarial Networks (GANs) to align the structure of the noise transition matrix with human priors more effectively and efficiently.

Empirical Validation

The paper conducts extensive experiments on standard benchmark datasets, CIFAR-10, CIFAR-100, and the industrial dataset Clothing1M, demonstrating the method's robustness across various noise structures: column-diagonal, tri-diagonal, and block-diagonal matrices. The results indicate significant performance enhancements over existing techniques, such as F-correction and S-adaptation, especially in cases with complex noise structures.

Numerical Highlights

On CIFAR-10 and CIFAR-100 datasets, Masking almost matches the performance of classifiers trained on clean data. The results for Clothing1M reflect a solid accuracy gain, highlighting its utility in practical, large-scale applications, despite only using approximated noise structures.

Practical and Theoretical Implications

The Masking approach emphasizes the value of cognitive insight in machine learning, demonstrating that human-inferred noise structures can guide the effective training of classifiers against label noise. In practice, this method can enhance the deployment of accurate models using noisy datasets, a common scenario in the collection of large-scale data from crowdsourced and automated systems.

Future Developments

The paper proposes future research directions, such as developing a self-correcting mechanism within Masking to dynamically adjust structure assumptions during training. Another avenue is exploring hybrid models that combine various data sources and human insights to improve noise modeling under diverse training conditions.

In conclusion, this paper advances the methodology for learning classifiers in noisy environments by effectively incorporating human cognitive biases into probabilistic models, showcasing promising improvements and setting a foundation for further exploration in adaptive learning frameworks.