Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Bayesian Hierarchical Probabilistic Forecasting of Intraday Electricity Prices (2403.05441v3)

Published 8 Mar 2024 in stat.AP and cs.LG

Abstract: We address the need for forecasting methodologies that handle large uncertainties in electricity prices for continuous intraday markets by incorporating parameter uncertainty and using a broad set of covariables. This study presents the first Bayesian forecasting of electricity prices traded on the German intraday market. Endogenous and exogenous covariables are handled via Orthogonal Matching Pursuit (OMP) and regularising priors. The target variable is the IDFull price index, with forecasts given as posterior predictive distributions. Validation uses the highly volatile 2022 electricity prices, which have seldom been studied. As a benchmark, we use all intraday transactions at the time of forecast to compute a live IDFull value. According to market efficiency, it should not be possible to improve on this last-price benchmark. However, we observe significant improvements in point measures and probability scores, including an average reduction of $5.9\,\%$ in absolute errors and an average increase of $1.7\,\%$ in accuracy when forecasting whether the IDFull exceeds the day-ahead price. Finally, we challenge the use of LASSO in electricity price forecasting, showing that OMP results in superior performance, specifically an average reduction of $22.7\,\%$ in absolute error and $20.2\,\%$ in the continuous ranked probability score.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Forecasting the intra-day spread densities of electricity prices. Energies 13. doi:10.3390/en13030687.
  2. Probabilistic price forecasting for day-ahead and intraday markets: Beyond the statistical model. Sustainability (Switzerland) 9. doi:10.3390/su9111990.
  3. Learning Probability Distributions of Day-Ahead Electricity Prices URL: https://arxiv.org/abs/2310.02867, doi:10.48550/ARXIV.2310.02867. publisher: arXiv Version Number: 2.
  4. JAX: composable transformations of Python+NumPy programs. URL: http://github.com/google/jax.
  5. Bayesian deep learning based method for probabilistic forecast of day-ahead electricity prices. Applied Energy 250, 1158–1175. URL: https://linkinghub.elsevier.com/retrieve/pii/S0306261919309237, doi:10.1016/j.apenergy.2019.05.068.
  6. Multivariate probabilistic forecasting of intraday electricity prices using normalizing flows. Applied Energy 346, 121370. URL: https://linkinghub.elsevier.com/retrieve/pii/S0306261923007341, doi:10.1016/j.apenergy.2023.121370.
  7. Exploratory Visual Analytics for the European Single Intra-Day Coupled Electricity Market, in: 2020 International Conference on Smart Energy Systems and Technologies (SEST), IEEE, Istanbul, Turkey. pp. 1–6. URL: https://ieeexplore.ieee.org/document/9203043/, doi:10.1109/SEST48500.2020.9203043.
  8. Comparing Predictive Accuracy. Technical Report t0169. National Bureau of Economic Research. Cambridge, MA. URL: http://www.nber.org/papers/t0169.pdf, doi:10.3386/t0169.
  9. TensorFlow Distributions URL: https://arxiv.org/abs/1711.10604, doi:10.48550/ARXIV.1711.10604. publisher: arXiv Version Number: 1.
  10. private communication. URL: https://www.epexspot.com/en/contact.
  11. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102, 359–378. URL: http://www.tandfonline.com/doi/abs/10.1198/016214506000001437, doi:10.1198/016214506000001437.
  12. From point forecasts to multivariate probabilistic forecasts: The Schaake shuffle for day-ahead electricity price forecasting. Energy Economics 120, 106602. URL: https://linkinghub.elsevier.com/retrieve/pii/S0140988323001007, doi:10.1016/j.eneco.2023.106602.
  13. Testing the equality of prediction mean squared errors. International Journal of Forecasting 13, 281–291. URL: https://linkinghub.elsevier.com/retrieve/pii/S0169207096007194, doi:10.1016/S0169-2070(96)00719-4.
  14. Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons. Statistical Science 35. URL: https://projecteuclid.org/journals/statistical-science/volume-35/issue-4/Best-Subset-Forward-Stepwise-or-Lasso-Analysis-and-Recommendations-Based/10.1214/19-STS733.full, doi:10.1214/19-STS733.
  15. Simulation-based Forecasting for Intraday Power Markets: Modelling Fundamental Drivers for Location, Shape and Scale of the Price Distribution , 1–42URL: http://arxiv.org/abs/2211.13002. arXiv: 2211.13002.
  16. Multivariate simulation‐based forecasting for intraday power markets: Modeling cross‐product price effects. Appl Stochastic Models Bus Ind. URL: https://onlinelibrary.wiley.com/doi/10.1002/asmb.2837, doi:10.1002/asmb.2837.
  17. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res. 15, 1593–1623.
  18. The effects of wind power on electricity markets: A case study of the Swedish intraday market. Energy Economics 96, 105159. URL: https://linkinghub.elsevier.com/retrieve/pii/S0140988321000645, doi:10.1016/j.eneco.2021.105159.
  19. Computing and Graphing Highest Density Regions. The American Statistician 50, 120–126. URL: http://www.tandfonline.com/doi/abs/10.1080/00031305.1996.10474359, doi:10.1080/00031305.1996.10474359.
  20. Estimating and Visualizing Conditional Densities. Journal of Computational and Graphical Statistics 5, 315–336. URL: http://www.tandfonline.com/doi/abs/10.1080/10618600.1996.10474715, doi:10.1080/10618600.1996.10474715.
  21. Forecasting the price distribution of continuous intraday electricity trading. Energies 12. doi:10.3390/en12224262.
  22. Modeling intraday markets under the new advances of the cross-border Intraday Project (XBID): Evidence from the German intraday market. Energies 12, 1–35. doi:10.3390/en12224339.
  23. The value of forecasts: Quantifying the economic gains of accurate quarter-hourly electricity price forecasts. Energy Economics 76, 411–423. URL: https://linkinghub.elsevier.com/retrieve/pii/S0140988318304018, doi:10.1016/j.eneco.2018.10.005.
  24. Optimal Order Execution in Intraday Markets: Minimizing Costs in Trade Trajectories URL: https://arxiv.org/abs/2009.07892, doi:10.48550/ARXIV.2009.07892. publisher: arXiv Version Number: 2.
  25. Deep distributional time series models and the probabilistic forecasting of intraday electricity prices. J of Applied Econometrics 38, 493–511. URL: https://onlinelibrary.wiley.com/doi/10.1002/jae.2959, doi:10.1002/jae.2959.
  26. Short-term electricity trading for system balancing: An empirical analysis of the role of intraday trading in balancing Germany’s electricity system. Renewable and Sustainable Energy Reviews 113, 109275. URL: https://linkinghub.elsevier.com/retrieve/pii/S1364032119304836, doi:10.1016/j.rser.2019.109275.
  27. Intraday Electricity Pricing of Night Contracts. Energies 13, 4501. URL: https://www.mdpi.com/1996-1073/13/17/4501, doi:10.3390/en13174501.
  28. An econometric model for intraday electricity trading. Phil. Trans. R. Soc. A. 379, 20190624. URL: https://royalsocietypublishing.org/doi/10.1098/rsta.2019.0624, doi:10.1098/rsta.2019.0624.
  29. Determining Fundamental Supply and Demand Curves in a Wholesale Electricity Market , 1–29URL: http://arxiv.org/abs/1903.11383. arXiv: 1903.11383.
  30. Integrated European intra-day electricity market: Rules, modeling and analysis. Applied Energy 238, 258–273. URL: https://linkinghub.elsevier.com/retrieve/pii/S0306261918318920, doi:10.1016/j.apenergy.2018.12.073.
  31. A Reinforcement Learning approach for the continuous electricity market of Germany: Trading from the perspective of a wind park operator. Energy and AI 8, 100139. URL: https://linkinghub.elsevier.com/retrieve/pii/S2666546822000039, doi:10.1016/j.egyai.2022.100139.
  32. Bayesian Predictive Distributions for Imbalance Prices With Time-Varying Factor Impacts. IEEE Trans. Power Syst. 38, 349–357. URL: https://ieeexplore.ieee.org/document/9761737/, doi:10.1109/TPWRS.2022.3165149.
  33. Performance of the autoregressive integrated moving average model with exogenous variables statistical model on the intraday market for the Denmark-West bidding area. Energy & Environment , 0958305X231199154URL: http://journals.sagepub.com/doi/10.1177/0958305X231199154, doi:10.1177/0958305X231199154.
  34. Day-ahead vs. Intraday—Forecasting the price spread to maximize economic benefits. Energies 12, 1–15. doi:10.3390/en12040631.
  35. Enhancing load, wind and solar generation for day-ahead forecasting of electricity prices. Energy Economics 99, 105273. URL: https://linkinghub.elsevier.com/retrieve/pii/S014098832100178X, doi:10.1016/j.eneco.2021.105273.
  36. Probabilistic forecasting with Factor Quantile Regression: Application to electricity trading URL: https://arxiv.org/abs/2303.08565, doi:10.48550/ARXIV.2303.08565. publisher: arXiv Version Number: 1.
  37. PCA forecast averaging - Predicting day-ahead and intraday electricity prices. Energies 13, 1–19. doi:10.3390/en13143530.
  38. Forecasting Electricity Prices, in: Oxford Research Encyclopedia of Economics and Finance. Oxford University Press. URL: http://arxiv.org/abs/2204.11735, doi:10.1093/acrefore/9780190625979.013.667.
  39. Distributional neural networks for electricity price forecasting. Energy Economics , 106843URL: http://arxiv.org/abs/2207.02832, doi:10.1016/j.eneco.2023.106843. arXiv: 2207.02832.
  40. Beating the naïve-combining lasso with naïve intraday electricity price forecasts. Energies 13, 1–16. doi:10.3390/en13071667.
  41. Statistical Rethinking: A Bayesian Course with Examples in R and Stan. 2 ed., Chapman and Hall/CRC. URL: https://www.taylorfrancis.com/books/9780429642319, doi:10.1201/9780429029608.
  42. Probabilistic Forecasting of German Electricity Imbalance Prices. Energies 15. doi:10.3390/en15144976. arXiv: 2205.11439.
  43. Estimation and simulation of the transaction arrival process in intraday electricity markets. Energies 12, 1–16. doi:10.3390/en12234518.
  44. Econometric modelling and forecasting of intraday electricity prices. Journal of Commodity Markets 19, 100107. URL: https://linkinghub.elsevier.com/retrieve/pii/S2405851319300728, doi:10.1016/j.jcomm.2019.100107.
  45. Ensemble forecasting for intraday electricity prices: Simulating trajectories. Applied Energy 279, 115801. URL: https://doi.org/10.1016/j.apenergy.2020.115801, doi:10.1016/j.apenergy.2020.115801.
  46. Recent advances in electricity price forecasting: A review of probabilistic forecasting. Renewable and Sustainable Energy Reviews 81, 1548–1568. URL: https://linkinghub.elsevier.com/retrieve/pii/S1364032117308808, doi:10.1016/j.rser.2017.05.234.
  47. Neural network based model comparison for intraday electricity price forecasting. Energies 12, 1–14. doi:10.3390/en12234557.
  48. Bayesian density forecasting of intraday electricity prices using multivariate skew t distributions. International Journal of Forecasting 24, 710–727. URL: https://linkinghub.elsevier.com/retrieve/pii/S0169207008001015, doi:10.1016/j.ijforecast.2008.08.009.
  49. Are fundamentals enough? Explaining price variations in the German day-ahead and intraday power market. Energy Economics 54, 376–387. URL: http://dx.doi.org/10.1016/j.eneco.2015.12.013, doi:10.1016/j.eneco.2015.12.013. publisher: Elsevier B.V.
  50. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition, in: Proceedings of 27th Asilomar conference on signals, systems and computers, IEEE. pp. 40–44.
  51. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12, 2825–2830.
  52. Influence of 15-minute contracts on frequency deviations and on the demand for balancing energy, in: International ETG Congress 2015; Die Energiewende - Blueprints for the new energy age, pp. 1–7.
  53. Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit. Technical Report. Citeseer.
  54. Towards the Prediction of Electricity Prices at the Intraday Market Using Shallow and Deep-Learning Methods. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12591 LNAI, 101–118. doi:10.1007/978-3-030-66981-2_9. iSBN: 9783030669805.
  55. Averaging Predictive Distributions Across Calibration Windows for Day-Ahead Electricity Price Forecasting. Energies 12, 2561. URL: https://www.mdpi.com/1996-1073/12/13/2561, doi:10.3390/en12132561.
  56. Analysing trading trends in continuous intraday electricity markets. 2021 56th International Universities Power Engineering Conference: Powering Net Zero Emissions, UPEC 2021 - Proceedings , 13–18doi:10.1109/UPEC50034.2021.9548168. publisher: IEEE ISBN: 9781665443890.
  57. False discoveries occur early on the Lasso path. The Annals of Statistics 45, 2133–2150. URL: https://projecteuclid.org/journals/annals-of-statistics/volume-45/issue-5/False-discoveries-occur-early-on-the-Lasso-path/10.1214/16-AOS1521.full, doi:10.1214/16-AOS1521.
  58. Regression Shrinkage and Selection Via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological) 58, 267–288. URL: https://rss.onlinelibrary.wiley.com/doi/10.1111/j.2517-6161.1996.tb02080.x, doi:10.1111/j.2517-6161.1996.tb02080.x.
  59. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit. IEEE Trans. Inform. Theory 53, 4655–4666. URL: http://ieeexplore.ieee.org/document/4385788/, doi:10.1109/TIT.2007.909108.
  60. Understanding intraday electricity markets: Variable selection and very short-term price forecasting using LASSO. International Journal of Forecasting 35, 1533–1547. URL: https://doi.org/10.1016/j.ijforecast.2019.02.001, doi:10.1016/j.ijforecast.2019.02.001. publisher: Elsevier B.V.
  61. Regularized quantile regression averaging for probabilistic electricity price forecasting. Energy Economics 95, 105121. URL: https://linkinghub.elsevier.com/retrieve/pii/S0140988321000268, doi:10.1016/j.eneco.2021.105121.
  62. State of the German Short-Term Power Market. Z Energiewirtsch 41, 87–103. URL: http://link.springer.com/10.1007/s12398-017-0196-9, doi:10.1007/s12398-017-0196-9.
  63. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17, 261–272. doi:10.1038/s41592-019-0686-2.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets