DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation (1803.10081v3)

Abstract: In computer vision, one is often confronted with problems of domain shifts, which occur when one applies a classifier trained on a source dataset to target data sharing similar characteristics (e.g. same classes), but also different latent data structures (e.g. different acquisition conditions). In such a situation, the model will perform poorly on the new data, since the classifier is specialized to recognize visual cues specific to the source domain. In this work we explore a solution, named DeepJDOT, to tackle this problem: through a measure of discrepancy on joint deep representations/labels based on optimal transport, we not only learn new data representations aligned between the source and target domain, but also simultaneously preserve the discriminative information used by the classifier. We applied DeepJDOT to a series of visual recognition tasks, where it compares favorably against state-of-the-art deep domain adaptation methods.

• The paper introduces DeepJDOT, which jointly optimizes deep feature alignment and class discrimination using optimal transport for unsupervised domain adaptation.
• It minimizes discrepancies by aligning CNN latent features and label distributions, effectively reducing domain shifts between source and target data.
• Empirical results on datasets like MNIST, USPS, and VisDA-2017 demonstrate DeepJDOT's superior performance over traditional domain adaptation methods.

Analysis of DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation

The paper under discussion presents DeepJDOT, an innovative methodology that addresses the unsupervised domain adaptation (UDA) issue in computer vision. UDA is crucial when classifiers trained on a source dataset encounter domain shifts upon application to a target dataset possessing analogous categories yet differing in underlying data structures. Conventional models lack the capability to generalize in such scenarios, thereby necessitating novel domain adaptation strategies like DeepJDOT.

Core Contribution

DeepJDOT proposes a framework that minimizes the discrepancy between joint deep feature representations and label distributions of source and target domains, utilizing optimal transport theory. Unlike conventional UDA techniques, DeepJDOT endeavors to mitigate data distribution shifts while simultaneously preserving class discriminative features through a joint optimization process. This dual-component optimization involves the adaptation of both feature extraction (embedding function) and classification concurrently in a convolutional neural network (CNN) paradigm. This integration allows the capture of semantic information within deeper layers, enhancing the alignment of representations between domains.

Methodological Details

Central to DeepJDOT is the coupling matrix $\gamma$ derived from Kantorovich’s optimal transport problem, representing a probabilistic connection between source and target distributions. DeepJDOT efficiently addresses two primary challenges in domain adaptation:

1. Scalability: By employing stochastic optimization and minibatch processing, the paper overcomes computational demands intrinsic to exhaustive optimal transport solutions.
2. Embedding Space: The proposed method evaluates cost functions within the CNN's latent feature layers rather than in the raw input space, achieving more robust and semantically meaningful alignment.

Additionally, DeepJDOT incorporates a source domain loss to counteract 'catastrophic forgetting', ensuring source domain performance is not compromised during adaptation.

Empirical Evaluation

DeepJDOT demonstrates superior performance across diverse domain adaptation tasks, including digit recognition tasks (e.g., MNIST to USPS) and the challenging Office-Home and VisDA-2017 datasets. The method consistently outperforms both traditional and contemporary domain adaptation methodologies, highlighting the effectiveness of coupling feature alignment with class discrimination under a unifying optimal transport framework.

For instance, on the VisDA-2017 dataset, DeepJDOT achieves a significant improvement on most class accuracy scores in comparison to other models, illustrating its capability to handle substantial domain shifts.

Implications and Future Prospects

The introduction of DeepJDOT marks an advancement in UDA, combining optimal transport with deep learning in a computationally feasible manner. The potential applications extend to any domain where acquiring labeled target data is challenging but large labeled source datasets are available. Theoretically, DeepJDOT provides insights into the merging of geometric aspects of data alignment with feature and label distributions in deep architectures.

Future research could explore extensions to more complex domain adaptation scenarios, including multi-target domains and continual learning setups. Another promising avenue is investigating advanced cost functions that leverage inter-layer feature semantics for enhanced performance.

In conclusion, DeepJDOT offers a robust and efficient framework for addressing domain adaptation issues in vision tasks, paving the way for more intelligent systems capable of operating under varied and previously unseen data environments.