Domain Adaptive Semantic Segmentation Using Weak Labels (2007.15176v2)

Abstract: Learning semantic segmentation models requires a huge amount of pixel-wise labeling. However, labeled data may only be available abundantly in a domain different from the desired target domain, which only has minimal or no annotations. In this work, we propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain. The weak labels may be obtained based on a model prediction for unsupervised domain adaptation (UDA), or from a human annotator in a new weakly-supervised domain adaptation (WDA) paradigm for semantic segmentation. Using weak labels is both practical and useful, since (i) collecting image-level target annotations is comparably cheap in WDA and incurs no cost in UDA, and (ii) it opens the opportunity for category-wise domain alignment. Our framework uses weak labels to enable the interplay between feature alignment and pseudo-labeling, improving both in the process of domain adaptation. Specifically, we develop a weak-label classification module to enforce the network to attend to certain categories, and then use such training signals to guide the proposed category-wise alignment method. In experiments, we show considerable improvements with respect to the existing state-of-the-arts in UDA and present a new benchmark in the WDA setting. Project page is at http://www.nec-labs.com/~mas/WeakSegDA.

• The paper presents a novel framework that leverages image-level weak labels to guide semantic segmentation across differing domains.
• It combines a weak-label classification module with category-wise adversarial feature alignment to reduce domain discrepancies.
• Experimental results on GTA5 to Cityscapes and SYNTHIA to Cityscapes transitions show significant improvements in intersection-over-union scores.

Domain Adaptive Semantic Segmentation Using Weak Labels

The paper "Domain Adaptive Semantic Segmentation Using Weak Labels" by Sujoy Paul et al. addresses the challenges of domain adaptation in semantic segmentation when abundant labeled data is primarily available in a source domain, differing significantly from an unlabeled or sparsely annotated target domain. Traditional supervised methods necessitate high-granularity, pixel-wise annotations, often impractical for real-world target domains due to prohibitive costs and resource constraints. This paper proposes leveraging weak supervision—namely, image-level annotations—in the target domain as a means to bridge this domain gap effectively.

Methodology

The authors introduce a novel framework that employs weak labels to facilitate domain adaptation. The core approach consists of two innovative components:

1. Weak-Label Classification Module: This module introduces an image-level classification task that guides the network to distinguish between the presence and absence of specific semantic categories within the target data. The classification leverages weak labels, which can either be pseudo labels generated through unsupervised techniques or acquired through limited human annotation, resulting in a weakly-supervised paradigm.
2. Category-Wise Feature Alignment: Using the weak labels, the paper proposes a mechanism for category-specific feature alignment between source and target domains. This is achieved by employing category-specific adversarial discriminators designed to align features within the context of identified categories, thereby enhancing domain adaptation performance.

Experimental Evaluation

The proposed methodology's effectiveness is demonstrated through experiments conducted on synthetic-to-real semantic segmentation tasks. Specifically, evaluations are performed on the GTA5 to Cityscapes and SYNTHIA to Cityscapes transitions, wherein the former synthetic datasets act as the source domain and Cityscapes as the target domain. These experiments validate significant performance improvements over existing unsupervised domain adaptation (UDA) methods.

• Numerical Results: The approach yielded substantial improvements in intersection-over-union (IoU) scores across multiple categories, confirming its efficacy. For example, the performance using pseudo-weak labels achieves a notable average IoU improvement by considerable margins on both datasets compared to the baseline methods.
• Utilization of Oracle-Weak Labels: Furthermore, by leveraging human-annotated weak labels (oracle-weak labels), the framework introduces a weakly-supervised domain adaptation (WDA) scenario that demonstrates even superior results, bridging the performance gap toward full supervision at minimal additional annotation cost.

Implications and Future Work

This research presents several implications for semantic segmentation, particularly in scenarios where acquiring exhaustive labeled datasets is untenable. The integration of weak labels aids in mitigating category-specific domain discrepancies, presenting a cost-effective alternative to complete re-annotation of target datasets.

Theoretically, this work underscores the viability of combining predictive pseudo-labeling methods with adversarial feature alignment strategies. Practically, it lays groundwork for more efficient training protocols that could be extended to other forms of weak supervision beyond image-level annotations, such as point-level or bounding box guidance.

As future avenues, exploration into more automated methods of acquiring reliable pseudo-weak labels could reduce dependence on human annotations, alongside adaptations of this approach for emerging domains, including video segmentation or other structured prediction tasks. The evolution of this method toward progressively learning domain-invariant features autonomously represents a promising trajectory in the field of computer vision.