2000 character limit reached
Multivariate Online Linear Regression for Hierarchical Forecasting (2402.14578v1)
Published 22 Feb 2024 in stat.ML, cs.LG, and math.OC
Abstract: In this paper, we consider a deterministic online linear regression model where we allow the responses to be multivariate. To address this problem, we introduce MultiVAW, a method that extends the well-known Vovk-Azoury-Warmuth algorithm to the multivariate setting, and show that it also enjoys logarithmic regret in time. We apply our results to the online hierarchical forecasting problem and recover an algorithm from this literature as a special case, allowing us to relax the hypotheses usually made for its analysis.
- Forecasting with temporal hierarchies. European Journal of Operational Research, 262(1):60–74, 2017.
- Relative loss bounds for on-line density estimation with the exponential family of distributions. Machine learning, 43:211–246, 2001.
- Minimax fixed-design linear regression. In Conference on Learning Theory, pp. 226–239. PMLR, 2015.
- Online hierarchical forecasting for power consumption data. arXiv preprint arXiv:2003.00585, 2020.
- Online hierarchical forecasting for power consumption data. International Journal of Forecasting, 38(1):339–351, 2022.
- Online state estimation for time-varying systems. IEEE Transactions on Automatic Control, 67(10):5424–5431, 2021.
- Prediction, learning, and games. Cambridge university press, 2006.
- Foster, D. P. Prediction in the worst case. The Annals of Statistics, pp. 1084–1090, 1991.
- Combining sku-level sales forecasts from models and experts. Expert Systems with Applications, 38(3):2365–2370, 2011. ISSN 0957-4174.
- Gaillard, P. Contributions to online robust aggregation: work on the approximation error and on probabilistic forecasting. applications to forecasting for energy markets. HAL, 2015, 2015.
- Uniform regret bounds over ℝnsuperscriptℝ𝑛\mathbb{R}^{n}blackboard_R start_POSTSUPERSCRIPT italic_n end_POSTSUPERSCRIPT for the sequential linear regression problem with the square loss. In Algorithmic Learning Theory, pp. 404–432. PMLR, 2019.
- Logarithmic regret algorithms for online convex optimization. Machine Learning, 69:169–192, 2007.
- Forecasting: principles and practice. OTexts, 2018.
- Dynamic structure embedded online multiple-output regression for streaming data. IEEE transactions on pattern analysis and machine intelligence, 41(2):323–336, 2018.
- A machine learning approach for forecasting hierarchical time series. Expert Systems with Applications, 182:115102, 2021.
- Sparse partial least squares regression for on-line variable selection with multivariate data streams. Statistical Analysis and Data Mining: The ASA Data Science Journal, 3(3):170–193, 2010.
- Orabona, F. A modern introduction to online learning. arXiv preprint arXiv:1912.13213, 2019.
- Stochastic online linear regression: the forward algorithm to replace ridge. Advances in Neural Information Processing Systems, 34:24430–24441, 2021.
- End-to-end learning of coherent probabilistic forecasts for hierarchical time series. In International Conference on Machine Learning, pp. 8832–8843. PMLR, 2021.
- Deep lstm-based transfer learning approach for coherent forecasts in hierarchical time series. Sensors, 21(13):4379, 2021.
- Online linear regression and its application to model-based reinforcement learning. Advances in Neural Information Processing Systems, 20, 2007.
- Coherent probabilistic forecasts for hierarchical time series. In International Conference on Machine Learning, pp. 3348–3357. PMLR, 2017.
- Vovk, V. Competitive on-line statistics. International Statistical Review, 69(2):213–248, 2001.
- Wintenberger, O. Optimal learning with bernstein online aggregation. Machine Learning, 106:119–141, 2017.
- Application of online multitask learning based on least squares support vector regression in the financial market. Applied Soft Computing, 121:108754, 2022.