DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks (1704.04110v3)

Abstract: Probabilistic forecasting, i.e. estimating the probability distribution of a time series' future given its past, is a key enabler for optimizing business processes. In retail businesses, for example, forecasting demand is crucial for having the right inventory available at the right time at the right place. In this paper we propose DeepAR, a methodology for producing accurate probabilistic forecasts, based on training an auto regressive recurrent network model on a large number of related time series. We demonstrate how by applying deep learning techniques to forecasting, one can overcome many of the challenges faced by widely-used classical approaches to the problem. We show through extensive empirical evaluation on several real-world forecasting data sets accuracy improvements of around 15% compared to state-of-the-art methods.

• The paper introduces DeepAR, a global autoregressive RNN that leverages inter-series data to achieve accurate probabilistic forecasting.
• It employs specific likelihood models, scale adjustments, and weighted sampling to handle diverse time series magnitudes and intermittent patterns.
• Empirical results show about a 15% improvement in forecast accuracy on real-world datasets, highlighting its robustness and scalability.

Essay on "DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks"

Introduction

The paper "DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks" by David Salinas, Valentin Flunkert, and Jan Gasthaus from Amazon Research addresses the recurring issue of probabilistic forecasting in the context of business processes. The primary contribution of the paper is the introduction of DeepAR, a methodology designed to produce accurate probabilistic forecasts by training an autoregressive recurrent neural network on multiple related time series. The approach contrasts significantly with traditional methods which often handle forecasting for individual time series separately. By leveraging data from multiple related series, DeepAR provides enhanced forecasting performance and theoretically grounded estimates of uncertainty.

Methodology

The DeepAR method builds upon the framework of autoregressive recurrent networks, wherein the key innovation is the simultaneous training on multiple time series data to exploit inter-series relationships. Using LSTM-based recurrent neural network architecture, DeepAR produces a global model enabled by the processing of historical data from all included time series. This method is tailored with specific modifications to handle issues such as diverse time series magnitudes and distributions, which are prevalent in real-world applications.

Key methodological elements of DeepAR include:

1. Likelihood Model: DeepAR incorporates a Gaussian likelihood for real-valued data and a negative Binomial likelihood for count data. The latter is integral for datasets with erratic, intermittent, or bursty patterns.
2. Scale Handling: To address the disparate scales of time series, DeepAR employs a scaling mechanism where autoregressive inputs are divided by a series-specific scale factor. This reduces the burden on the RNN to manage scale internally and improves learning efficiency.
3. Weighted Sampling: Given the power-law distribution observed in datasets such as retail sales, DeepAR uses a strategy where the probability of selecting a training instance is proportional to its velocity (average value). This helps to counteract the imbalance caused by having a few high-velocity time series interspersed among many low-velocity ones.

Empirical Evaluation

The authors conduct extensive empirical evaluations using several real-world datasets, including retail sales data from Amazon, energy consumption data, and traffic occupancy. Across these datasets:

• DeepAR showed approximately a 15% improvement in forecasting accuracy over state-of-the-art methods.
• Models trained using DeepAR were capable of learning complex seasonality patterns and handling items with little or no historical data, demonstrating the method's robustness and general applicability.
• In terms of scalability, DeepAR handled large datasets effectively, confirming its viability for deployment in real-world forecasting scenarios.

Numerical Results

Numerical results presented in the paper are compelling. For instance:

• On datasets exhibiting a power-law distribution of sales velocities, such as Amazon's retail data, DeepAR demonstrated superior performance in terms of forecast accuracy and probabilistic calibration.
• The use of the negative Binomial likelihood allowed DeepAR to model non-Gaussian distribution properties accurately, significantly outperforming traditional methods that assume Gaussian errors.

Implications and Future Work

The practical implications of this research are far-reaching. Accurate probabilistic forecasts hold immense value in numerous domains, including inventory management, resource allocation, and supply chain optimization. The introduction of DeepAR affords businesses the ability to make more informed, data-driven decisions under uncertainty.

Theoretically, this research contributes to the growing evidence that deep learning methods can outperform classical techniques in time series forecasting by leveraging large datasets and learning complex patterns automatically. This stands in contrast to existing skepticism stemming from mixed results in earlier research on neural networks for forecasting.

Future developments could include exploring additional likelihood models for different types of time series data, enhancing real-time learning capabilities, and optimizing computational efficiency for even larger datasets. Furthermore, expanding the interpretability of DeepAR's forecasts could better support decision-makers in practical applications.

Conclusion

The DeepAR methodology presents a significant advancement in probabilistic forecasting, highlighting the advantages of deep learning techniques applied to large-scale time series data. Its robust handling of varying magnitudes, seasonality, and uncertainty growth, coupled with empirical validation across diverse datasets, underscores its potential for broad application in business forecasting and beyond. The research sets a precedent for further exploration and development in the intersection of recurrent neural networks and time series forecasting.