Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation (2111.12940v2)

Abstract: Self-training has greatly facilitated domain adaptive semantic segmentation, which iteratively generates pseudo labels on unlabeled target data and retrains the network. However, realistic segmentation datasets are highly imbalanced, pseudo labels are typically biased to the majority classes and basically noisy, leading to an error-prone and suboptimal model. In this paper, we propose a simple region-based active learning approach for semantic segmentation under a domain shift, aiming to automatically query a small partition of image regions to be labeled while maximizing segmentation performance. Our algorithm, Region Impurity and Prediction Uncertainty (RIPU), introduces a new acquisition strategy characterizing the spatial adjacency of image regions along with the prediction confidence. We show that the proposed region-based selection strategy makes more efficient use of a limited budget than image-based or point-based counterparts. Further, we enforce local prediction consistency between a pixel and its nearest neighbors on a source image. Alongside, we develop a negative learning loss to make the features more discriminative. Extensive experiments demonstrate that our method only requires very few annotations to almost reach the supervised performance and substantially outperforms state-of-the-art methods. The code is available at https://github.com/BIT-DA/RIPU.

• The paper introduces Region Impurity and Prediction Uncertainty (RIPU), a novel region-based active learning method that selects informative regions based on impurity and uncertainty to reduce annotation costs for domain adaptive semantic segmentation.
• Extensive experiments show RIPU achieves performance close to fully supervised methods with minimal annotations, outperforming existing methods on benchmarks like GTAV to Cityscapes and SYNTHIA to Cityscapes.
• The RIPU framework offers significant practical value by reducing annotation costs for tasks like autonomous driving and provides a theoretical foundation for developing more context-aware active learning systems by capitalizing on spatial data.

Overview of Active Learning for Domain Adaptive Semantic Segmentation

The paper "Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation" proposes a novel active learning framework aimed at improving domain adaptive semantic segmentation via fewer annotations. Current self-training methods in domain adaptation retrain segmentation networks using pseudo labels generated from high-confidence predictions on target domains. However, issues arise due to class imbalance and pseudo-label noise, leading to suboptimal segmentation performance on the target domain. This paper introduces Region Impurity and Prediction Uncertainty (RIPU), a region-based active learning method designed to address these challenges by selecting informative image regions for annotation, thus minimizing labeling costs while maximizing segmentation performance.

Key Contributions

1. Region-Based Selection Strategy: The paper introduces a novel acquisition strategy that leverages region impurity and prediction uncertainty. The approach focuses on querying small partitions of an image that are both spatially diverse and uncertain in prediction output. This strategy effectively utilizes the annotation budget compared to traditional image-based or point-based selection strategies.
2. Local Prediction Consistency and Negative Learning: In addition to region-based active learning, the method enhances model training via local prediction consistency and a negative learning loss function. Local consistency enforces stable predictions across spatially adjacent pixels, while the negative learning loss helps to refine model representations by focusing on absent classes during adaptation.
3. Comprehensive Experiments and Results: Extensive experimental evaluations demonstrate that RIPU requires minimal target domain annotations to achieve performance levels close to fully supervised models, substantially outperforming existing state-of-the-art domain adaptation and active learning methods.

Experimental Insights

The paper conducts experiments on prominent synthetic-to-real domain adaptation benchmarks such as GTAV to Cityscapes and SYNTHIA to Cityscapes, employing DeepLab-v2 and DeepLab-v3+ as backbone architectures. The proposed methods are rigorously compared against leading self-training and active learning techniques. Results show that RIPU not only enhances mean Intersection-over-Union (mIoU) metrics significantly but also improves performance on rare or small object classes. This is largely due to its balanced and diverse querying strategy that emphasizes spatially rich regions, which are overlooked by baseline approaches.

Practical and Theoretical Implications

Practically, this active learning framework demonstrates how minimal human intervention can achieve competitive performance with lower annotation costs, which is invaluable for applications like autonomous driving and medical analysis where data labeling is expensive. Theoretically, the integration of region impurity and uncertainty offers a fresh perspective on sampling strategies by capitalizing on the spatial contiguity of pixel data, suggesting a pathway for the development of more informed and context-aware active learning systems.

Future Directions

The exploration of region-based active learning strategies serves as a catalyst for future research in AI focusing on label efficiency and domain adaptation. Potential areas of exploration include:

• Extending the approach to other dense prediction tasks like object detection and instance segmentation.
• Investigating the impact of various superpixel segmentation algorithms as a foundation for generating informative regions.
• Exploring the applicability of RIPU within a semi-supervised or unsupervised domain adaptation framework, possibly integrating with generative models for enhanced synthetic data utility.

In conclusion, the paper presents a methodologically sound and practically impactful approach to semantic segmentation, leveraging active learning techniques to bridge the domain gap with minimal annotations. It sets a precedent for future methodological advancements and applications in efficient model training across domains with limited labeled data.