Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis (1806.08946v1)

Abstract: Recent years have witnessed the unprecedented rising of time series from almost all kindes of academic and industrial fields. Various types of deep neural network models have been introduced to time series analysis, but the important frequency information is yet lack of effective modeling. In light of this, in this paper we propose a wavelet-based neural network structure called multilevel Wavelet Decomposition Network (mWDN) for building frequency-aware deep learning models for time series analysis. mWDN preserves the advantage of multilevel discrete wavelet decomposition in frequency learning while enables the fine-tuning of all parameters under a deep neural network framework. Based on mWDN, we further propose two deep learning models called Residual Classification Flow (RCF) and multi-frequecy Long Short-Term Memory (mLSTM) for time series classification and forecasting, respectively. The two models take all or partial mWDN decomposed sub-series in different frequencies as input, and resort to the back propagation algorithm to learn all the parameters globally, which enables seamless embedding of wavelet-based frequency analysis into deep learning frameworks. Extensive experiments on 40 UCR datasets and a real-world user volume dataset demonstrate the excellent performance of our time series models based on mWDN. In particular, we propose an importance analysis method to mWDN based models, which successfully identifies those time-series elements and mWDN layers that are crucially important to time series analysis. This indeed indicates the interpretability advantage of mWDN, and can be viewed as an indepth exploration to interpretable deep learning.

• The paper introduces mWDN, a novel framework that integrates trainable wavelet decomposition with deep learning to analyze time series in both time and frequency domains.
• It employs multilevel decomposition to power advanced models like Residual Classification Flow and multi-frequency LSTM, achieving competitive results on 40 UCR datasets.
• The study demonstrates that frequency-aware components improve interpretability by identifying critical model layers, paving the way for applications in finance, healthcare, and beyond.

Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysis

The paper introduces a novel approach to improve time series analysis by integrating wavelet decomposition within a deep learning framework, thus facilitating frequency-aware modeling. This is achieved through the Multilevel Wavelet Decomposition Network (mWDN), which characterizes the time-series data in both time and frequency domains, enabling the parameter fine-tuning typically available in deep learning models.

Overview of mWDN Framework

mWDN leverages Multilevel Discrete Wavelet Decomposition (MDWD) to analyze time-series data, providing a multi-resolution perspective. Unlike traditional MDWD that operates with fixed parameters, mWDN allows all parameters to be trainable, optimizing the model for specific tasks. Importantly, the paper highlights that mWDN retains the advantages of wavelet-based decomposition, enhancing interpretability without forfeiting the deep learning model’s adaptability and predictive capabilities. This network structure is pivotal for two main applications: time series classification (TSC) and time series forecasting (TSF).

Deep Learning Models Based on mWDN

Two specific models are developed using mWDN: Residual Classification Flow (RCF) for TSC, and multi-frequency Long Short-Term Memory (mLSTM) for TSF. The RCF model benefits from residual learning techniques to extract diverse features from multilevel wavelet-decomposed sub-series, advancing its ability to classify time series. In contrast, mLSTM enhances TSF by focusing on how different frequency components predict future trends. Through end-to-end training which includes mWDN components, these models seamlessly integrate time-frequency analysis, showcasing improved performance metrics.

Experimental Validation and Performance

The evaluation of mWDN-based models on 40 UCR datasets showed competitive results, underscoring the utility of frequency components in improving classification accuracy. Specifically, RCF demonstrated superior classification performance as compared to existing benchmarks, affirming the effectiveness of integrating frequency decomposition into classification tasks. Similarly, mLSTM's application to a user volume dataset in Wuxi demonstrated lower prediction errors in TSF scenarios, reinforcing the advantage of a frequency-aware approach.

Importance Analysis and Interpretability

The paper introduces an importance analysis method for assessing the significance of different elements and layers within mWDN-based models, addressing the interpretability challenge in deep learning. This analysis facilitates the identification of critical components that drive the model's predictions by evaluating sensitivity across model layers. The findings from this method substantiate the interpretability-enhancing role of wavelet decomposition in distinguishing between informative features across different frequency bands.

Implications and Future Directions

The integration of wavelet decomposition into deep learning models marks a significant step in evolving time series methodologies, particularly in terms of interpretability and performance. Practically, the ability to discern frequency-specific information extends the utility of deep learning models in domains requiring nuanced time-series analyses, such as finance and healthcare. Theoretically, this paper paves the way for future work in integrating other signal processing techniques with deep learning, potentially offering more granular insights into time-series behavior.

In conclusion, the multilevel wavelet decomposition network exemplifies a promising advancement in establishing frequency-aware deep learning models for time series analysis. Future explorations could leverage similar integrations to enhance model interpretability across different applications, broadening the horizons of AI in both academic and industrial contexts.