UniTS: A Unified Multi-Task Time Series Model (2403.00131v3)

Abstract: Although pre-trained transformers and reprogrammed text-based LLMs have shown strong performance on time series tasks, the best-performing architectures vary widely across tasks, with most models narrowly focused on specific areas, such as time series forecasting. Unifying predictive and generative time series tasks within a single model remains challenging. We introduce UniTS, a unified multi-task time series model that utilizes task tokenization to integrate predictive and generative tasks into a single framework. UniTS employs a modified transformer block to capture universal time series representations, enabling transferability from a heterogeneous, multi-domain pre-training dataset-characterized by diverse dynamic patterns, sampling rates, and temporal scales-to a wide range of downstream datasets with varied task specifications and data domains. Tested on 38 datasets across human activity sensors, healthcare, engineering, and finance, UniTS achieves superior performance compared to 12 forecasting models, 20 classification models, 18 anomaly detection models, and 16 imputation models, including adapted text-based LLMs. UniTS also demonstrates strong few-shot and prompt capabilities when applied to new domains and tasks. In single-task settings, UniTS outperforms competitive task-specialized time series models. Code and datasets are available at https://github.com/mims-harvard/UniTS.

• The paper presents a unified task specification that eliminates task-specific modules and achieves a 10.5% gain in zero-shot forecasting accuracy.
• It employs sequence and variable attention with a dynamic linear operator to efficiently handle diverse time series data shapes.
• It outperforms specialized models in 27 out of 38 tasks, improving few-shot imputation MSE by 12.4% over the best baseline.

UniTS: Building a Unified Time Series Model

The paper entitled "UniTS: Building a Unified Time Series Model" presents a sophisticated approach towards developing a versatile time series model capable of handling multiple diverse tasks without the reliance on task-specific modules. This research addresses a notable gap in the domain, where general-purpose models for time series analysis have not been extensively explored.

Overview

UniTS is designed as a foundational time series model analogous to those used in language and vision domains. The model integrates a universal task specification which supports tasks such as classification, forecasting, imputation, and anomaly detection. It achieves this through a unified network backbone, utilizing sequence and variable attention, accompanied by a dynamic linear operator. This design supports universal adaptability and outperforms task-specific models on 38 multi-domain datasets.

Key Contributions

1. Universal Task Specification: The model leverages a novel prompting framework, converting diverse tasks into a unified token representation. This allows for efficient task specification and adaptability across various tasks without the need for specialized modules.
2. Data-Domain Agnostic Network: Using self-attention mechanisms across both sequence and variable dimensions, UniTS accommodates varying data shapes typical of time series data. The introduction of a dynamic linear operator further enhances its ability to model dense relations within sequences of any length.
3. Unified Model with Shared Weights: UniTS maintains shared weights across all tasks, enhancing generalization. A unified masked reconstruction pretraining scheme bolsters the model's capability to manage generative and recognition tasks concurrently.

Numerical Results

The UniTS model demonstrates superior performance over existing models across multiple datasets. It shows notable prowess in zero-shot and prompt-based learning scenarios. For instance, UniTS achieved 10.5% improvement in one-step zero-shot forecasting over the leading baseline model, highlighting its effective handling of new forecasting horizons and variable numbers.

In multi-task settings, the UniTS model outperformed task-specific baselines in 27 out of 38 tasks, showcasing its adaptability and generalization capabilities. For few-shot transfer learning, UniTS improved MSE by 12.4% on imputation tasks compared to the best baseline.

Implications and Future Directions

The development of UniTS signifies an important stride towards generalist models in time series analysis, supporting diverse data domains and task specifications with a single, unified architecture. The implications for practical applications are vast, including improved scalability and efficiency in deploying time series solutions across industries such as healthcare, finance, and engineering.

Future research could explore scaling the model further or exploring its efficacy in additional domains. Moreover, investigating how UniTS can integrate with emerging data types or combine with reinforcement learning for real-time applications could prove beneficial.

Conclusion

The paper's introduction of UniTS marks a significant advancement in the pursuit of unified time series models. By eliminating reliance on task-specific modules, UniTS propels the potential for universal, adaptable, and efficient time series analysis, paving the way for innovative applications and methodologies in the field of artificial intelligence.