Bidirectional Learning for Domain Adaptation of Semantic Segmentation (1904.10620v1)

Abstract: Domain adaptation for semantic image segmentation is very necessary since manually labeling large datasets with pixel-level labels is expensive and time consuming. Existing domain adaptation techniques either work on limited datasets, or yield not so good performance compared with supervised learning. In this paper, we propose a novel bidirectional learning framework for domain adaptation of segmentation. Using the bidirectional learning, the image translation model and the segmentation adaptation model can be learned alternatively and promote to each other. Furthermore, we propose a self-supervised learning algorithm to learn a better segmentation adaptation model and in return improve the image translation model. Experiments show that our method is superior to the state-of-the-art methods in domain adaptation of segmentation with a big margin. The source code is available at https://github.com/liyunsheng13/BDL.

Bidirectional Learning for Domain Adaptation of Semantic Segmentation

The paper "Bidirectional Learning for Domain Adaptation of Semantic Segmentation" addresses the challenge of adapting semantic segmentation models to different domains without requiring labeled data from the target domain. The traditional approach of manually labeling datasets is costly and time-consuming. This work introduces a novel bidirectional learning framework that enhances the performance of semantic segmentation through a closed-loop interaction between two models: an image translation model and a segmentation adaptation model.

The proposed framework facilitates unsupervised domain adaptation by employing a bidirectional learning process that iteratively updates both models. This process consists of two directions: forward, "translation-to-segmentation," and backward, "segmentation-to-translation." The forward direction is responsible for training the segmentation adaptation model using translated images, while the backward direction improves the image translation model using the updated segmentation results. This closed-loop learning approach allows for mutual enhancement between the two models, gradually reducing the domain gap.

Key contributions of the paper include:

1. Bidirectional Learning System: The method introduces a bidirectional learning architecture that enables mutual learning between the image translation and segmentation adaptation models, forming a closed loop that refines both models iteratively.
2. Self-Supervised Learning (SSL): The paper proposes a self-supervised learning mechanism to refine the segmentation adaptation model. This approach incrementally aligns the source and target domains by leveraging pseudo labels with high confidence, which are generated without target domain annotations.
3. Perceptual Loss for Image Translation: The research introduces a novel perceptual loss that ensures semantic consistency between the original and translated images, enhancing the image translation quality and supporting better segmentation adaptation.

Experiments demonstrate the efficacy of the proposed framework by outperforming state-of-the-art methodologies on large-scale datasets, specifically in adapting from synthetic datasets like SYNTHIA and GTA5 to real-world datasets such as Cityscapes. The proposed bidirectional learning achieves notable improvements over traditional sequential methods, with strong numerical results indicating substantial enhancement in mean Intersection over Union (mIoU) scores.

The implications of this work are significant for both theoretical exploration and practical application. By constructing a more efficient domain adaptation framework, this research provides insights into how iterative interaction between different models can be leveraged for better learning outcomes. The paper sets a foundation for future exploration in domain adaptation, potentially expanding into other areas of computer vision and beyond.

The results suggest promising pathways for further research in AI, including exploration of more complex domain shifts and integration with other learning paradigms. Future work may focus on refining the bidirectional approach, experimenting with different architectures for the models involved, and extending the methodology to other challenging tasks beyond semantic segmentation.