Maximum Classifier Discrepancy for Unsupervised Domain Adaptation (1712.02560v4)

Abstract: In this work, we present a method for unsupervised domain adaptation. Many adversarial learning methods train domain classifier networks to distinguish the features as either a source or target and train a feature generator network to mimic the discriminator. Two problems exist with these methods. First, the domain classifier only tries to distinguish the features as a source or target and thus does not consider task-specific decision boundaries between classes. Therefore, a trained generator can generate ambiguous features near class boundaries. Second, these methods aim to completely match the feature distributions between different domains, which is difficult because of each domain's characteristics. To solve these problems, we introduce a new approach that attempts to align distributions of source and target by utilizing the task-specific decision boundaries. We propose to maximize the discrepancy between two classifiers' outputs to detect target samples that are far from the support of the source. A feature generator learns to generate target features near the support to minimize the discrepancy. Our method outperforms other methods on several datasets of image classification and semantic segmentation. The codes are available at \url{https://github.com/mil-tokyo/MCD_DA}

• The paper introduces a novel unsupervised domain adaptation technique using task-specific classifiers to guide feature alignment.
• It employs a three-step training process that involves initial training, discrepancy maximization, and feature alignment through discrepancy minimization.
• Experimental results demonstrate substantial improvements in classification and segmentation tasks, validating the method's theoretical and practical significance.

Maximum Classifier Discrepancy for Unsupervised Domain Adaptation

The paper "Maximum Classifier Discrepancy for Unsupervised Domain Adaptation," authored by Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada from The University of Tokyo and RIKEN, introduces a novel approach to unsupervised domain adaptation (UDA) by leveraging task-specific decision boundaries for aligning distributions of source and target features.

Introduction and Motivation

The paper articulates two major issues with existing adversarial learning methods for domain adaptation:

1. Ambiguous Features Near Class Boundaries: Traditional domain classifier-based methods do not account for task-specific decision boundaries. Consequently, the feature generator may produce ambiguous features near class boundaries, adversely affecting classification accuracy.
2. Complete Distribution Matching: Aiming to completely align feature distributions between source and target domains is often impractical due to inherent differences in domain characteristics.

To address these problems, the authors introduce a method that leverages the discrepancies between two task-specific classifiers as a signal to guide the feature generator. This approach uses decision boundaries to detect target samples that are far from the source distribution's support, thereby refining the feature generator.

Methodology

The proposed method involves a three-step training process:

1. Initial Training: Both the classifiers and the generator are trained on source samples to ensure discriminative features.
2. Discrepancy Maximization: The classifiers are trained to maximize the discrepancy of their predictions on target samples for a fixed generator.
3. Discrepancy Minimization: The feature generator is trained to minimize this discrepancy, effectively aligning the target features with the source features.

This iterative adversarial training cycle encourages the generator to produce discriminative features for target samples, thereby enhancing adaptation performance.

Theoretical Insights

The paper connects its method to the theory proposed by Ben-David et al., which bounds the target error using the divergence of distributions. The introduced measure, $\mathcal{H} \Delta \mathcal{H}$-distance, approximated by the discrepancy between classifier outputs, serves as an intuitive and theoretically sound mechanism for domain adaptation.

Experimental Results

Classification Tasks

The effectiveness of the proposed method is evaluated on several domain adaptation tasks, including:

• Digit Classification (SVHN to MNIST, SYN SIGNS to GTSRB, MNIST to USPS):
  • On SVHN to MNIST, the proposed method achieves an impressive 96.2% accuracy, outperforming other state-of-the-art methods.
• Object Classification (VisDA):
  • The method exhibits remarkable performance, achieving a mean accuracy of 71.9%, significantly higher than baseline domain adaptation methods like MMD and DANN.

The results consistently demonstrate that considering task-specific decision boundaries in domain adaptation improves performance markedly across diverse tasks.

Semantic Segmentation

The paper extends the methodology to semantic segmentation tasks using synthetic datasets (GTA5, Synthia) as the source and the Cityscapes dataset as the target:

• GTA5 to Cityscapes: The proposed method achieves a mean IoU of up to 39.9 with the DRN-D-105 architecture, surpassing other methods in most categories.
• Synthia to Cityscapes: The performance consistently improves with the proposed method when compared to non-adapted and baseline adapted models.

These experiments validate that the approach is effective for complex, high-dimensional tasks like semantic segmentation.

Implications and Future Directions

The implications of this research are manifold:

• Practical: The proposed method offers a robust solution for domain adaptation, particularly useful in scenarios where labeled target data is scarce or unavailable.
• Theoretical: The connection to $\mathcal{H} \Delta \mathcal{H}$-distance provides a solid theoretical foundation, encouraging further theoretical exploration in domain adaptation.

Future research can explore several directions, such as:

• Extending the method to other architectures and tasks to generalize its applicability.
• Investigating the integration with other domain adaptation strategies, such as cycle-consistent adversarial networks.

Conclusion

This paper presents an innovative method for unsupervised domain adaptation by leveraging task-specific classifiers to maximize and subsequently minimize the discrepancy in their predictions on target samples. The approach shows superior performance across different computer vision tasks, providing both empirical and theoretical contributions to the field of domain adaptation.