Deep Unsupervised Domain Adaptation for Time Series Classification: a Benchmark (2312.09857v2)

Abstract: Unsupervised Domain Adaptation (UDA) aims to harness labeled source data to train models for unlabeled target data. Despite extensive research in domains like computer vision and natural language processing, UDA remains underexplored for time series data, which has widespread real-world applications ranging from medicine and manufacturing to earth observation and human activity recognition. Our paper addresses this gap by introducing a comprehensive benchmark for evaluating UDA techniques for time series classification, with a focus on deep learning methods. We provide seven new benchmark datasets covering various domain shifts and temporal dynamics, facilitating fair and standardized UDA method assessments with state of the art neural network backbones (e.g. Inception) for time series data. This benchmark offers insights into the strengths and limitations of the evaluated approaches while preserving the unsupervised nature of domain adaptation, making it directly applicable to practical problems. Our paper serves as a vital resource for researchers and practitioners, advancing domain adaptation solutions for time series data and fostering innovation in this critical field. The implementation code of this benchmark is available at https://github.com/EricssonResearch/UDA-4-TSC.

• The paper establishes a benchmark by introducing seven diverse time series datasets to evaluate deep unsupervised domain adaptation methods.
• It employs techniques like DANN, CDAN, OTDA, and layer transfer to realign models for improved performance across domains.
• The study demonstrates that effective hyperparameter tuning, particularly using target risk, substantially boosts classifier performance over architecture choices.

Overview of Unsupervised Domain Adaptation for Time Series Classification

Unsupervised Domain Adaptation (UDA) is a critical subfield in machine learning, where the goal is to enable a model, trained on a labeled dataset from one domain, to perform well on a different, unlabeled domain that shares some similarities with the training domain. While UDA is well-studied in certain areas like Natural Language Processing and image recognition, it isn't as explored in the domain of time series classification—which is used in a variety of applications such as healthcare, finance, and activity recognition.

A paper fills this gap by establishing a benchmark for UDA in time series classification, which aims to provide a standardized way to evaluate new UDA methods. To facilitate this, the researchers introduced seven new datasets to the community that vary in domain shifts and temporal dynamics.

Domain Adaptation Techniques

The paper discusses several strategies:

1. Transfer of specific layers from Neural Networks, where certain layers trained on the source domain are reused in the target domain.
2. Domain-Adversarial Neural Network (DANN) seeks to minimize domain differences through adversarial training.
3. Conditional Adversarial Domain Adaptation (CDAN) aligns conditional distributions to achieve domain invariance.
4. Optimal Transport Domain Adaptation (OTDA) uses transport theory to align source and target distributions.

These methodologies attempt to realign the models to work well with the target domain that lacks labels, addressing issues arising from distribution shifts between the training and target domains.

Hyperparameter Tuning without Labels

A notable challenge in UDA is ensuring the model is well-tuned in the absence of labeled target data, which is a common practice in supervised learning. The researchers investigate three standard approaches:

• Target Risk: Involves using (at least some) target labels, serving more as an upper bound on performance.
• Source Risk: Relies on empirical risk based on source labels.
• Importance Weighted Cross-Validation (IWCV): Considers the potential shift between source and target domain distributions.

Experimentation and Findings

The experimental framework for assessing UDA methods involved using various hyperparameter tuning techniques and deep UDA algorithms, equipped with advanced architectures like InceptionTime's backbone.

The results demonstrate that the right choice of hyperparameter tuning method, such as the Target Risk approach, can lead to a significant improvement in classifier performance. Additionally, the paper discovers that while the choice of neural network architecture is crucial, the UDA technique itself seems to have a more notable impact on performance.

The Implications of the Study

Setting up this benchmark and proposing new datasets paves the way for more rigorous and fair evaluation of UDA approaches in time series data. With the provision of standardized datasets and methods, future work can build upon these findings to develop more sophisticated UDA algorithms, ultimately advancing the field and its applications.

These advancements could prove invaluable in real-world scenarios where acquiring labeled data is costly or impractical, such as in various sensor-based monitoring systems or patient health tracking where the model deployment environment is ever-changing. The paper emphasizes the ever-present need for domain adaptation solutions in dynamic settings that require machine learning models to be adaptive and resilient to changes in data distribution.