Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 41 tok/s
Gemini 2.5 Pro 46 tok/s Pro
GPT-5 Medium 21 tok/s Pro
GPT-5 High 20 tok/s Pro
GPT-4o 91 tok/s Pro
Kimi K2 178 tok/s Pro
GPT OSS 120B 474 tok/s Pro
Claude Sonnet 4 38 tok/s Pro
2000 character limit reached

A probabilistic forecast methodology for volatile electricity prices in the Australian National Electricity Market (2311.07289v2)

Published 13 Nov 2023 in cs.LG

Abstract: The South Australia region of the Australian National Electricity Market (NEM) displays some of the highest levels of price volatility observed in modern electricity markets. This paper outlines an approach to probabilistic forecasting under these extreme conditions, including spike filtration and several post-processing steps. We propose using quantile regression as an ensemble tool for probabilistic forecasting, with our combined forecasts achieving superior results compared to all constituent models. Within our ensemble framework, we demonstrate that averaging models with varying training length periods leads to a more adaptive model and increased prediction accuracy. The applicability of the final model is evaluated by comparing our median forecasts with the point forecasts available from the Australian NEM operator, with our model outperforming these NEM forecasts by a significant margin.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. AEMO (2021a). Pre-dispatch. Technical Report. URL: https://www.aemo.com.au/-/media/files/electricity/nem/5ms/procedures-workstream/stakeholder-consultation/dispatch-procedures/so_op_3704---predispatch---marked-up.pdf.
  2. AEMO (2021b). Spot market operations timetable. URL: https://www.aemo.com.au/-/media/files/electricity/nem/security_and_reliability/dispatch/spot-market-operations-timetable.pdf.
  3. AEMO (2022). Market price cap. URL: https://www.aemc.gov.au/news-centre/media-releases/2022-23-market-price-cap-now-available.
  4. AEMO (2023). Constraint formulation guidelines. URL: https://aemo.com.au/-/media/files/stakeholder_consultation/consultations/nem-consultations/2022/cfg-and-scvpf/final/constraint-formulation-guidelines-v12---final_.pdf.
  5. Probabilistic deep neural network price forecasting based on residential load and wind speed predictions. IET Renewable Power Generation, 13, 1840–1848. URL: https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/iet-rpg.2018.6257. doi:https://doi.org/10.1049/iet-rpg.2018.6257. arXiv:https://ietresearch.onlinelibrary.wiley.com/doi/pdf/10.1049/iet-rpg.2018.6257.
  6. Ausgrid (). Network prices. URL: https://www.ausgrid.com.au/Industry/Regulation/Network-prices.
  7. Ausgrid (2012). Solar home electricity data. URL: https://www.ausgrid.com.au/Industry/Our-Research/Data-to-share/Solar-home-electricity-data.
  8. Combining day-ahead forecasts for british electricity prices. Energy Economics, 35, 88--103. doi:10.1016/j.eneco.2011.12.001.
  9. Increasing the reliability of reliability diagrams. Weather and Forecasting, 22, 651 -- 661. URL: https://journals.ametsoc.org/view/journals/wefo/22/3/waf993_1.xml. doi:https://doi.org/10.1175/WAF993.1.
  10. Conditional density forecast of electricity price based on ensemble elm and logistic emos. IEEE Transactions on Smart Grid, 10, 3031--3043. doi:10.1109/TSG.2018.2817284.
  11. Short-term electricity prices forecasting based on support vector regression and auto-regressive integrated moving average modeling. Energy Conversion and Management, 51, 1911--1917. URL: https://www.sciencedirect.com/science/article/pii/S0196890410000890. doi:https://doi.org/10.1016/j.enconman.2010.02.023.
  12. Electricity price forecasting for operational scheduling of behind-the-meter storage systems. IEEE Transactions on Smart Grid, 9, 6612--6622. doi:10.1109/TSG.2017.2717282.
  13. Forecasting electricity prices for a day-ahead pool-based electric energy market. International Journal of Forecasting, 21, 435--462. URL: https://www.sciencedirect.com/science/article/pii/S0169207004001311. doi:https://doi.org/10.1016/j.ijforecast.2004.12.005.
  14. A two-stage forecasting model: Exponential smoothing and multiple regression. Management Science, 13, B--501.
  15. Comparing predictive accuracy. Journal of Business & Economic Statistics, 13, 253--263. doi:10.1080/07350015.1995.10524599.
  16. Optimal sizing and scheduling of community battery storage within a local market. In Proceedings of the Thirteenth ACM International Conference on Future Energy Systems e-Energy ’22 (p. 34–46). New York, NY, USA: Association for Computing Machinery. URL: https://doi.org/10.1145/3538637.3538837. doi:10.1145/3538637.3538837.
  17. Support vector regression machines, . 9. URL: https://proceedings.neurips.cc/paper/1996/file/d38901788c533e8286cb6400b40b386d-Paper.pdf.
  18. Electric, A. (). Amber electric. URL: https://www.amber.com.au/.
  19. Additive models and robust aggregation for gefcom2014 probabilistic electric load and electricity price forecasting. International Journal of Forecasting, 32, 1038--1050. URL: https://www.sciencedirect.com/science/article/pii/S0169207015001545. doi:https://doi.org/10.1016/j.ijforecast.2015.12.001.
  20. Understanding the fine structure of electricity prices. The Journal of Business, 79, 1225--1261. URL: http://www.jstor.org/stable/10.1086/500675.
  21. A review of uncertainty representations and metaverification of uncertainty assessment techniques for renewable energies. Renewable and Sustainable Energy Reviews, 96, 352--379.
  22. Probabilistic forecasts, calibration and sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69, 243--268. URL: https://doi.org/10.1111/j.1467-9868.2007.00587.x. doi:10.1111/j.1467-9868.2007.00587.x. arXiv:https://academic.oup.com/jrsssb/article-pdf/69/2/243/49794500/jrsssb_69_2_243.pdf.
  23. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102, 359--378. URL: https://doi.org/10.1198/016214506000001437. doi:10.1198/016214506000001437.
  24. Improved methods of combining forecasts. Journal of forecasting, 3, 197--204.
  25. Hersbach, H. (2000). Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting, 15, 559 -- 570. URL: https://journals.ametsoc.org/view/journals/wefo/15/5/1520-0434_2000_015_0559_dotcrp_2_0_co_2.xml. doi:https://doi.org/10.1175/1520-0434(2000)015<0559:DOTCRP>2.0.CO;2.
  26. Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond. International Journal of Forecasting, 32, 896--913. URL: https://www.sciencedirect.com/science/article/pii/S0169207016000133. doi:https://doi.org/10.1016/j.ijforecast.2016.02.001.
  27. Energy forecasting: A review and outlook. IEEE Open Access Journal of Power and Energy, 7, 376--388. doi:10.1109/OAJPE.2020.3029979.
  28. Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting. International Journal of Forecasting, 35, 1389--1399.
  29. A note on averaging day-ahead electricity price forecasts across calibration windows. IEEE Transactions on Sustainable Energy, 10, 321--323. doi:10.1109/TSTE.2018.2869557.
  30. Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling. Energy Economics, 38, 96--110. URL: https://www.sciencedirect.com/science/article/pii/S0140988313000625. doi:https://doi.org/10.1016/j.eneco.2013.03.013.
  31. Conformal prediction interval estimation and applications to day-ahead and intraday power markets. International Journal of Forecasting, 37, 777--799. URL: https://www.sciencedirect.com/science/article/pii/S0169207020301473. doi:https://doi.org/10.1016/j.ijforecast.2020.09.006.
  32. Quantile regression. Journal of economic perspectives, 15, 143--156.
  33. Probabilistic electricity price forecasting with bayesian stochastic volatility models. Energy Economics, 80, 610--620. URL: https://www.sciencedirect.com/science/article/pii/S0140988319300544. doi:https://doi.org/10.1016/j.eneco.2019.02.004.
  34. Kupiec, P. (1995). Techniques for verifying the accuracy of risk measurement models. Finance and Economics Discussion Series 95-24 Board of Governors of the Federal Reserve System (U.S.). URL: https://EconPapers.repec.org/RePEc:fip:fedgfe:95-24.
  35. Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark. Applied Energy, 293, 116983. URL: https://www.sciencedirect.com/science/article/pii/S0306261921004529. doi:https://doi.org/10.1016/j.apenergy.2021.116983.
  36. Verification of solar irradiance probabilistic forecasts. Solar Energy, 194, 254--271. URL: https://www.sciencedirect.com/science/article/pii/S0038092X19310382. doi:https://doi.org/10.1016/j.solener.2019.10.041.
  37. Probabilistic load forecasting considering temporal correlation: Online models for the prediction of households’ electrical load. Applied Energy, 303, 117594. URL: https://www.sciencedirect.com/science/article/pii/S0306261921009685. doi:https://doi.org/10.1016/j.apenergy.2021.117594.
  38. A hybrid model for gefcom2014 probabilistic electricity price forecasting. International Journal of Forecasting, 32, 1051--1056. URL: https://www.sciencedirect.com/science/article/pii/S0169207015001430. doi:https://doi.org/10.1016/j.ijforecast.2015.11.008.
  39. Probabilistic forecasting of electricity spot prices using factor quantile regression averaging. International Journal of Forecasting, 32, 957--965. URL: https://www.sciencedirect.com/science/article/pii/S0169207014001848. doi:https://doi.org/10.1016/j.ijforecast.2014.12.004.
  40. The m4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36, 54--74. URL: https://www.sciencedirect.com/science/article/pii/S0169207019301128. doi:https://doi.org/10.1016/j.ijforecast.2019.04.014. M4 Competition.
  41. Selection of calibration windows for day-ahead electricity price forecasting. Energies, 11, 2364. doi:10.3390/en11092364.
  42. Quantile regression forests. Journal of Machine Learning Research, 7.
  43. An empirical comparison of alternative schemes for combining electricity spot price forecasts. Energy Economics, 46, 395--412. URL: https://www.sciencedirect.com/science/article/pii/S0140988314001716. doi:https://doi.org/10.1016/j.eneco.2014.07.014.
  44. Computing electricity spot price prediction intervals using quantile regression and forecast averaging. Computational Statistics, 30, 791--803. URL: https://doi.org/10.1007/s00180-014-0523-0. doi:10.1007/s00180-014-0523-0.
  45. Recent advances in electricity price forecasting: A review of probabilistic forecasting. Renewable and Sustainable Energy Reviews, 81, 1548--1568. URL: https://www.sciencedirect.com/science/article/pii/S1364032117308808. doi:https://doi.org/10.1016/j.rser.2017.05.234.
  46. Online, W. W. (). Historical weather forecast data. URL: https://www.worldweatheronline.com/.
  47. Reliability diagrams for non-parametric density forecasts of continuous variables: Accounting for serial correlation. Quarterly Journal of the Royal Meteorological Society, 136, 77--90. doi:https://doi.org/10.1002/qj.559.
  48. Non-parametric probabilistic forecasts of wind power: required properties and evaluation. Wind Energy, 10, 497--516. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/we.230. doi:https://doi.org/10.1002/we.230. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/we.230.
  49. Forecasting day-ahead electricity prices: Utilizing hourly prices. Energy Economics, 50, 227--239.
  50. Averaging predictive distributions across calibration windows for day-ahead electricity price forecasting. Energies, 12, 2561. doi:10.3390/en12132561.
  51. Tesla (). Tesla powerwall. URL: https://www.tesla.com/en_au/powerwall.
  52. Regularized quantile regression averaging for probabilistic electricity price forecasting. Energy Economics, 95, 105121. URL: https://www.sciencedirect.com/science/article/pii/S0140988321000268. doi:https://doi.org/10.1016/j.eneco.2021.105121.
  53. Combining probabilistic load forecasts. IEEE Transactions on Smart Grid, 10, 3664--3674. doi:10.1109/TSG.2018.2833869.
  54. Weron, R. (2014). Electricity price forecasting: A review of the state-of-the-art with a look into the future. International Journal of Forecasting, 30, 1030--1081. URL: https://www.sciencedirect.com/science/article/pii/S0169207014001083. doi:https://doi.org/10.1016/j.ijforecast.2014.08.008.
  55. Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models. International Journal of Forecasting, 24, 744--763. URL: https://www.sciencedirect.com/science/article/pii/S0169207008000952. doi:https://doi.org/10.1016/j.ijforecast.2008.08.004. Energy Forecasting.
  56. Economic impact of electricity market price forecasting errors: A demand-side analysis. IEEE Transactions on Power Systems, 25, 254--262. doi:10.1109/TPWRS.2009.2030380.
  57. A statistical approach for interval forecasting of the electricity price. IEEE Transactions on Power Systems, 23, 267--276. doi:10.1109/TPWRS.2008.919309.
Citations (11)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now