Importance Weighted Adversarial Nets for Partial Domain Adaptation (1803.09210v2)

Abstract: This paper proposes an importance weighted adversarial nets-based method for unsupervised domain adaptation, specific for partial domain adaptation where the target domain has less number of classes compared to the source domain. Previous domain adaptation methods generally assume the identical label spaces, such that reducing the distribution divergence leads to feasible knowledge transfer. However, such an assumption is no longer valid in a more realistic scenario that requires adaptation from a larger and more diverse source domain to a smaller target domain with less number of classes. This paper extends the adversarial nets-based domain adaptation and proposes a novel adversarial nets-based partial domain adaptation method to identify the source samples that are potentially from the outlier classes and, at the same time, reduce the shift of shared classes between domains.

• The paper introduces an importance weighted adversarial network framework that selectively reweights source samples to focus on shared classes between domains.
• It employs two domain classifiers to mitigate domain shift and reduce the Jensen-Shannon divergence between reweighted source and target distributions.
• Empirical results on benchmark datasets demonstrate significant performance gains over traditional methods, emphasizing both theoretical and practical advances.

Importance Weighted Adversarial Nets for Partial Domain Adaptation

The paper "Importance Weighted Adversarial Nets for Partial Domain Adaptation" presents a novel framework designed to address the challenges inherent in partial domain adaptation. This scenario arises when the target domain includes fewer classes than the source domain, thus diverging from the traditional domain adaptation assumption where identical label spaces are a prerequisite. The authors extend current adversarial domain adaptation techniques to accommodate the partial domain adaptation paradigm by integrating an importance weighting mechanism.

Key Contributions and Methodology

The paper makes significant contributions by proposing a method that leverages importance weighted adversarial networks to tackle partial domain adaptation. The core idea revolves around identifying and re-weighting source samples—particularly those from classes absent in the target domain—to ensure that the adaptation process primarily focuses on shared classes.

1. Reweighting Scheme: The authors introduce a strategic reweighting mechanism applied to source samples. This mechanism is guided by the outputs of an adversarial network, which provides an indication of whether a source sample belongs to an outlier class. The motivation behind this strategy is to mitigate the effect of domain shift by prioritizing samples from classes present in both source and target domains.
2. Two Domain Classifiers: A distinctive aspect of the proposed solution is the employment of two domain classifiers. The first classifier, applied to the source samples, determines their importance by classifying the likelihood they belong to the target domain. The resulting weights then inform the second classifier, which processes both weighted source samples and target samples.
3. Theoretical Validation: The authors provide theoretical underpinnings demonstrating that their method effectively reduces the Jensen-Shannon divergence between the reweighted source distribution and the target distribution, thereby justifying the adaptation model from an optimization standpoint.
4. Entropy Minimization: The model also incorporates an entropy minimization term to preserve the target domain data structure. This approach aligns the target features with those learned from the source data while supporting decision boundaries that delineate distinct class regions.

Empirical Evaluation

The proposed method was empirically validated on several benchmark datasets, including Office+Caltech-10, Office-31, and Caltech256-Office10. The results indicate a substantial improvement over traditional domain adaptation techniques like AlexNet fine-tuning, with the method outperforming by a notable margin. Additionally, the approach compares favorably with state-of-the-art partial transfer methods, particularly SAN, delivering competitive accuracy with fewer parameters.

Practical and Theoretical Implications

The practical implications of this research are significant, given the real-world presence of differing label spaces between source and target domains. The framework enhances the robustness and applicability of domain adaptation solutions across diverse application scenarios where the target domain is inherently smaller or less diverse. Theoretically, the introduction of a dual-classifier approach and an importance weighting scheme refines the understanding of adversarial networks' role in domain adaptation, specifically when applied to tasks necessitating selective class-focus.

Future Directions

Future work may expand upon this framework by considering larger-scale datasets or varying class imbalances within the target domain. Moreover, investigating the integration of other domain adaptation techniques alongside importance-weighted adversarial strategies could yield further improvements in model performance and domain alignment capabilities.

In conclusion, the paper presents a sophisticated and well-substantiated approach to a nuanced problem in domain adaptation. The importance weighted adversarial network provides a flexible and practical solution for partial domain adaptation, positioning itself as a valuable model for scenarios demanding selective class adaptation.