Chronos: Learning the Language of Time Series (2403.07815v3)

Abstract: We introduce Chronos, a simple yet effective framework for pretrained probabilistic time series models. Chronos tokenizes time series values using scaling and quantization into a fixed vocabulary and trains existing transformer-based LLM architectures on these tokenized time series via the cross-entropy loss. We pretrained Chronos models based on the T5 family (ranging from 20M to 710M parameters) on a large collection of publicly available datasets, complemented by a synthetic dataset that we generated via Gaussian processes to improve generalization. In a comprehensive benchmark consisting of 42 datasets, and comprising both classical local models and deep learning methods, we show that Chronos models: (a) significantly outperform other methods on datasets that were part of the training corpus; and (b) have comparable and occasionally superior zero-shot performance on new datasets, relative to methods that were trained specifically on them. Our results demonstrate that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks, positioning pretrained models as a viable tool to greatly simplify forecasting pipelines.

• The paper introduces Chronos, which adapts transformer-based T5 models to forecast time series by converting continuous data into discrete tokens.
• It leverages data augmentation techniques like TSMix and KernelSynth to pretrain on varied domains, enhancing performance in both in-domain and zero-shot scenarios.
• The results demonstrate Chronos as a unified forecasting tool that challenges legacy methods through its robust generalization across unseen datasets.

Chronos: Simplifying Time Series Forecasting with Pretrained LLMs

Introduction to Chronos

In recent times, the landscape of time series forecasting has been continually evolving, with an increasing shift toward leveraging the advancements in deep learning and LLMs. The newly introduced framework, Chronos, exemplifies this transition by employing a LLMing approach to predict future values in time series data. The essence of Chronos lies in its simple yet powerful methodology that adapts conventional transformer-based LLMs, specifically the T5 family, for time series forecasting tasks.

Methodology

Tokenization and Model Architecture

A key distinction between time series and natural language data is their representation; while the former is typically continuous and real-valued, the latter comprises discrete tokens. Chronos tackles this by scaling and quantizing time series data into a fixed vocabulary, effectively transforming time series into a sequence of discrete tokens. This tokenization process enables the application of off-the-shelf LLMs to time series forecasting without necessitating alterations in model architecture. The choice of uniform binning for tokenization further ensures that Chronos can handle a wide array of time series patterns, including trends and seasonality.

Training Procedure

Chronos models are pretrained on a comprehensive dataset amalgamated from a variety of sources, covering domains as diverse as energy, finance, healthcare, and climate science, among others. To enhance the robustness and generalization capability of the models, data augmentation strategies like TSMix and KernelSynth play a crucial role. TSMix, specifically, generates new time series via convex combinations of randomly selected base time series, whereas KernelSynth leverages Gaussian processes to produce synthetic time series data. These procedures significantly augment the diversity of the training corpus, thereby augmenting model performance on both seen and unseen datasets.

Evaluation and Results

Chronos models were benchmarked against a suite of both classical statistical methods and contemporary deep learning models across two principal benchmarks: in-domain, where models were evaluated on datasets included in their training corpus; and zero-shot, testing the models' performance on entirely unseen datasets. The results establish Chronos as a remarkably effective framework, outperforming traditional methods and showcasing competitive accuracy relative to domain-specific deep learning models in the in-domain benchmark. More notably, in the zero-shot setting, Chronos models demonstrated a unique proficiency to generalize across unseen time series data, highlighting its promise as a versatile forecasting tool.

Implications and Future Directions

The promising outcomes of Chronos beckon a plethora of implications and potential future explorations. Practically, Chronos challenges the status quo of developing and deploying distinct forecasting models for each unique time series task, presenting a unified model capable of delivering precise forecasts across diverse domains and datasets. Theoretically, it raises pertinent questions about the underlying similarities between language and time series data and how generic modeling approaches can be adeptly applied to seemingly distinct data types.

Looking ahead, avenues for enhancing Chronos include exploring refined tokenization schemes to further optimize performance, extending the framework to accommodate multivariate time series and incorporate external covariates, and potentially employing Chronos' learned representations for tasks beyond forecasting, such as anomaly detection and series classification.

Conclusion

Chronos represents a significant stride toward simplifying time series forecasting by leveraging the power of pretrained LLMs. Its success not only underscores the versatility of LLMs but also sets a foundational step toward developing more generalized, efficient, and accessible forecasting tools that can adapt seamlessly across varied temporal dynamics observed in real-world phenomena.