Adaptive Early-Learning Correction for Segmentation from Noisy Annotations (2110.03740v2)

Abstract: Deep learning in the presence of noisy annotations has been studied extensively in classification, but much less in segmentation tasks. In this work, we study the learning dynamics of deep segmentation networks trained on inaccurately-annotated data. We discover a phenomenon that has been previously reported in the context of classification: the networks tend to first fit the clean pixel-level labels during an "early-learning" phase, before eventually memorizing the false annotations. However, in contrast to classification, memorization in segmentation does not arise simultaneously for all semantic categories. Inspired by these findings, we propose a new method for segmentation from noisy annotations with two key elements. First, we detect the beginning of the memorization phase separately for each category during training. This allows us to adaptively correct the noisy annotations in order to exploit early learning. Second, we incorporate a regularization term that enforces consistency across scales to boost robustness against annotation noise. Our method outperforms standard approaches on a medical-imaging segmentation task where noises are synthesized to mimic human annotation errors. It also provides robustness to realistic noisy annotations present in weakly-supervised semantic segmentation, achieving state-of-the-art results on PASCAL VOC 2012. Code is available at https://github.com/Kangningthu/ADELE

• The paper introduces ADELE, which adaptively identifies the onset of noise memorization in each class using IoU curves.
• It applies multiscale consistency regularization to maintain stable predictions across different image scales.
• Empirical validation on medical imaging and PASCAL VOC 2012 datasets shows superior segmentation accuracy over existing methods.

Adaptive Early-Learning Correction for Segmentation from Noisy Annotations

The paper presents an investigation and subsequent methodological innovation focused on overcoming challenges with deep learning segmentation tasks when trained on noisily annotated data. While the impact of noisy annotations has been extensively studied in classification domains, segmentation exhibits distinct dynamics that require targeted solutions. Specifically, the research highlights how segmentation networks initially learn from clean pixel-level labels during an "early-learning" phase but eventually overfit and memorize erroneous annotations. This memorization varies across different semantic categories, which motivates a novel method named ADELE (ADaptive Early-Learning corrEction) to tackle this issue.

Key Aspects of ADELE

1. Class-wise Adaptive Correction: The ADELE method notably recognizes that memorization does not occur uniformly across semantic categories. Therefore, it implements an adaptive approach where the onset of memorization is detected separately for each category using the Intersection over Union (IoU) curves during training. By identifying these inflection points, ADELE is able to preemptively adjust its training strategy and correct noisy labels by leveraging the existing model output.
2. Consistency Regularization: ADELE incorporates a multiscale consistency regularization term, which demands that predictions remain consistent across different scales of input images. This aspect fortifies the network against noise, promoting robustness by providing additional supervision signals that discourage overfitting to noisy labels. These measures collectively enable more accurate segmentation outcomes when compared to existing standard approaches.
3. Empirical Validation: The effectiveness of ADELE is substantiated through its performance on two datasets: a medical imaging dataset (SegTHOR) where annotation noise is synthetically introduced, and the PASCAL VOC 2012 dataset for weakly-supervised semantic segmentation (WSSS). In both instances, ADELE demonstrates superior resilience to noise, evidenced by substantial gains in segmentation accuracy compared to both baseline models and state-of-the-art alternatives. Notably, on PASCAL VOC 2012, ADELE achieves state-of-the-art results, reinforcing its applicability to weakly supervised conditions.

Implications and Future Directions

The ADELE approach has broad implications for domains reliant on pixel-level image annotations, where acquiring high-quality labels is often resource-intensive or prone to human error. Specifically, fields such as medical imaging, where expert annotation is both critical and challenging, stand to benefit significantly from techniques that improve data robustness and reduce memorization artifacts. The research suggests practical pathways to integrate adaptive label correction in various domains, leveraging early-learning behavior characteristic of neural networks.

Moreover, this paper sets the stage for further exploration of adaptive correction mechanisms and consistency regularization. Future work could probe into more complex adaptive strategies that account for other factors such as inter-object relationships and contextual information in images. Additionally, extending this framework to real-world noisy datasets beyond synthetic approximations can help validate the robustness and generalizability of the ADELE framework more comprehensively.

In summary, the paper meaningfully advances the field of noisy annotation learning in segmentation tasks, providing a foundational step towards more resilient, accurate, and intelligent training paradigms under practical data limitations.