Benchmarking Multivariate Time Series Classification Algorithms

Abstract: Time Series Classification (TSC) involved building predictive models for a discrete target variable from ordered, real valued, attributes. Over recent years, a new set of TSC algorithms have been developed which have made significant improvement over the previous state of the art. The main focus has been on univariate TSC, i.e. the problem where each case has a single series and a class label. In reality, it is more common to encounter multivariate TSC (MTSC) problems where multiple series are associated with a single label. Despite this, much less consideration has been given to MTSC than the univariate case. The UEA archive of 30 MTSC problems released in 2018 has made comparison of algorithms easier. We review recently proposed bespoke MTSC algorithms based on deep learning, shapelets and bag of words approaches. The simplest approach to MTSC is to ensemble univariate classifiers over the multivariate dimensions. We compare the bespoke algorithms to these dimension independent approaches on the 26 of the 30 MTSC archive problems where the data are all of equal length. We demonstrate that the independent ensemble of HIVE-COTE classifiers is the most accurate, but that, unlike with univariate classification, dynamic time warping is still competitive at MTSC.

• The paper benchmarks MTSC algorithms on 30 UEA datasets, showing that traditional distance-based methods like DTW variants remain strong baselines.
• The study finds that shapelet-based classifiers and bag-of-words models, such as WEASEL+MUSE, deliver competitive performance, especially on high-dimensional data.
• The research indicates that deep learning models capture complex temporal patterns but often underperform simpler methods, suggesting new directions for algorithmic improvement.

Benchmarking Multivariate Time Series Classification Algorithms

The paper "Benchmarking Multivariate Time Series Classification Algorithms" by Alejandro Pasos Ruiz, Michael Flynn, and Anthony Bagnall provides a comprehensive analysis of the efficacy of various algorithms for Multivariate Time Series Classification (MTSC). The focus is on benchmarking new algorithmic proposals against established methods using the University of East Anglia (UEA) archive of 30 MTSC problems. This study is crucial given the increasing prevalence of MTSC tasks in real-world applications such as human activity recognition and physiological signal analysis, where each case is characterized by multiple time series indexed by a common label.

Overview of Approaches

The authors categorize MTSC algorithms into several classes: distance-based methods, including Multiple Dynamic Time Warping (DTW), shapelet-based classifiers, bag-of-words models, and deep learning frameworks.

1. Distance-Based Methods:
   • Dynamic Time Warping (DTW) remains a prominent distance function for TSC, owing to its ability to compensate for temporal distortions in time series. The study explores variants like independent (DTW_I) and dependent (DTW_D) DTW, along with an adaptive DTW (DTW_A) that combines both.
2. Shapelet-Based Classifiers:
   • The study examines the Generalized Random Shapelet Forest (gRFS), which uses randomization to introduce variability and reduce the training time normally associated with shapelet methods.
3. Bag-of-Words Models:
   • The WEASEL+MUSE algorithm, a hybrid approach that enhances univariate methods for the multivariate domain using Multivariate Unsupervised Symbols and dErivatives (MUSE), shows promising results but is constrained by memory consumption.
4. Deep Learning-Based Approaches:
   • The MLSTM-FCN (or MLCN) and Time Series Attentional Prototype Network (TapNet) are notable deep learning architectures tailored for MTSC, leveraging both convolutional and LSTM layers.

Experimental Setup and Results

The authors conduct their experiments using the UEA MTSC archive, which comprises 26 datasets of equal-length series. They employ various open-source software tools like the tsml and aeon toolkits, with specific experiments maximizing hardware and time resources, thus highlighting practical implementation considerations.

The experimental results reveal that:

• HIVE-COTE with component classifiers treats each dimension independently and achieves superior performance, being significantly more accurate than DTW_D on a number of benchmarks.
• Shapelet-based methods, notably gRFS and STC, are competitive, especially on datasets with a high number of dimensions or longer time series.
• Deep learning models, although typically effective in capturing complex temporal patterns, underperform relative to simpler distance-based methods, indicating potential areas for further algorithmic refinement.

Implications and Future Directions

This comparative analysis underscores the complexity and needs for specialized approaches in MTSC. The study indicates that traditional methods like DTW continue to serve as meaningful baselines, and the potential for newer algorithms to outperform them lies in effectively capturing inter-dimensional dependencies and balancing computational requirements.

In the context of real-world applications, the research suggests avenues for improving MTSC algorithms by integrating dimensional relations explicitly and optimizing for both accuracy and resource constraints.

Looking ahead, the expansion of the UEA MTSC archive and ensuing research will be instrumental in bridging the gap between univariate and multivariate TSC, fostering advancements in methodological frameworks that efficiently harness the rich information present in multivariate datasets.

The paper also suggests future work should focus on creating algorithms that integrate these dimensions directly, moving beyond the currently employed adaptations of univariate classifiers for multivariate problems.