SkyGPT: Probabilistic Short-term Solar Forecasting Using Synthetic Sky Videos from Physics-constrained VideoGPT (2306.11682v1)
Abstract: In recent years, deep learning-based solar forecasting using all-sky images has emerged as a promising approach for alleviating uncertainty in PV power generation. However, the stochastic nature of cloud movement remains a major challenge for accurate and reliable solar forecasting. With the recent advances in generative artificial intelligence, the synthesis of visually plausible yet diversified sky videos has potential for aiding in forecasts. In this study, we introduce \emph{SkyGPT}, a physics-informed stochastic video prediction model that is able to generate multiple possible future images of the sky with diverse cloud motion patterns, by using past sky image sequences as input. Extensive experiments and comparison with benchmark video prediction models demonstrate the effectiveness of the proposed model in capturing cloud dynamics and generating future sky images with high realism and diversity. Furthermore, we feed the generated future sky images from the video prediction models for 15-minute-ahead probabilistic solar forecasting for a 30-kW roof-top PV system, and compare it with an end-to-end deep learning baseline model SUNSET and a smart persistence model. Better PV output prediction reliability and sharpness is observed by using the predicted sky images generated with SkyGPT compared with other benchmark models, achieving a continuous ranked probability score (CRPS) of 2.81 (13\% better than SUNSET and 23\% better than smart persistence) and a Winkler score of 26.70 for the test set. Although an arbitrary number of futures can be generated from a historical sky image sequence, the results suggest that 10 future scenarios is a good choice that balances probabilistic solar forecasting performance and computational cost.
- IPCC. IPCC Special Report on Renewable Energy Sources and Climate Change Mitigation (Chapter 9). Prepared by Working Group III of the Intergovernmental Panel on Climate Change. Cambridge University Press, 2011. URL http://www.ipcc-wg3.de/srren-report/.
- Yuchi Sun. Short-term Solar Forecast Using Convolutional Neural Networks with Sky Images. PhD thesis, 2019. URL https://www.proquest.com/dissertations-theses/short-term-solar-forecast-using-convolutional/docview/2466057410/se-2.
- Review on probabilistic forecasting of photovoltaic power production and electricity consumption. Renewable and Sustainable Energy Reviews, 81:1484–1512, 2018.
- Intra-hour forecasting with a total sky imager at the uc san diego solar energy testbed. Solar Energy, 85(11):2881–2893, 2011.
- Intra-hour dni forecasting based on cloud tracking image analysis. Solar Energy, 91:327–336, 2013.
- Cloud-tracking methodology for intra-hour DNI forecasting. Solar Energy, 102:267–275, April 2014. ISSN 0038-092X. doi: 10.1016/j.solener.2014.01.030.
- Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning. Solar Energy, 98:592–603, December 2013. ISSN 0038-092X. doi: 10.1016/j.solener.2013.10.020.
- Real-time prediction intervals for intra-hour DNI forecasts. Renewable Energy, 83:234–244, 2015a. ISSN 18790682. doi: 10.1016/j.renene.2015.04.022. URL http://dx.doi.org/10.1016/j.renene.2015.04.022.
- Short-term reforecasting of power output from a 48 MWe solar PV plant. Solar Energy, 112:68–77, feb 2015b. ISSN 0038-092X. doi: 10.1016/J.SOLENER.2014.11.017. URL https://www.sciencedirect.com/science/article/pii/S0038092X14005611.
- Adaptive image features for intra-hour solar forecasts. Journal of Renewable and Sustainable Energy, 11(3):036101, May 2019. doi: 10.1063/1.5091952.
- Short-term solar power forecast with deep learning: Exploring optimal input and output configuration. Solar Energy, 188:730–741, aug 2019. doi: 10.1016/j.solener.2019.06.041. URL https://linkinghub.elsevier.com/retrieve/pii/S0038092X19306164.
- PV power output prediction from sky images using convolutional neural network: The comparison of sky-condition-specific sub-models and an end-to-end model. Journal of Renewable and Sustainable Energy, 12(4):046101, jul 2020. doi: 10.1063/5.0014016. URL http://aip.scitation.org/doi/10.1063/5.0014016.
- SolarNet: A sky image-based deep convolutional neural network for intra-hour solar forecasting. Solar Energy, 204(April):71–78, 2020. doi: 10.1016/j.solener.2020.03.083. URL https://doi.org/10.1016/j.solener.2020.03.083.
- Convolutional Neural Networks Applied to Sky Images for Short-Term Solar Irradiance Forecasting. In EU PVSEC, pages 1834 – 1837, 2020. ISBN 3-936338-73-6. doi: 10.4229/EUPVSEC20202020-6BV.5.15. URL https://www.eupvsec-proceedings.com/proceedings?paper=49346.
- Resampling and data augmentation for short-term pv output prediction based on an imbalanced sky images dataset using convolutional neural networks. Solar Energy, 224:341–354, 2021.
- Convolutional neural networks for intra-hour solar forecasting based on sky image sequences. Applied Energy, 310:118438, 2022.
- Deep photovoltaic nowcasting. Solar Energy, 176(September):267–276, 2018. doi: 10.1016/j.solener.2018.10.024. URL https://doi.org/10.1016/j.solener.2018.10.024.
- Benchmarking of deep learning irradiance forecasting models from sky images – An in-depth analysis. Solar Energy, 224:855–867, August 2021. ISSN 0038-092X. doi: 10.1016/j.solener.2021.05.056. URL https://linkinghub.elsevier.com/retrieve/pii/S0038092X21004266.
- ECLIPSE: Envisioning CLoud Induced Perturbations in Solar Energy. Applied Energy, 326:119924, November 2022. ISSN 0306-2619. doi: 10.1016/j.apenergy.2022.119924.
- Vincent Le Guen and Nicolas Thome. A deep physical model for solar irradiance forecasting with fisheye images. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2020-June:2685–2688, 2020a. ISSN 21607516. doi: 10.1109/CVPRW50498.2020.00323.
- Digital particle image velocimetry. Experiments in fluids, 10(4):181–193, 1991.
- The computation of optical flow. ACM computing surveys (CSUR), 27(3):433–466, 1995.
- An ensemble prediction intervals approach for short-term pv power forecasting. Solar Energy, 155:1072–1083, 2017.
- Short-term solar irradiance forecasting using calibrated probabilistic models. NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning, 2020.
- Omnivision forecasting: Combining satellite and sky images for improved deterministic and probabilistic intra-hour solar energy predictions. Applied Energy, 336:120818, April 2023. ISSN 0306-2619. doi: 10.1016/j.apenergy.2023.120818.
- A Review on Deep Learning Techniques for Video Prediction. apr 2020. URL http://arxiv.org/abs/2004.05214.
- Attention is all you need. Advances in neural information processing systems, 30, 2017.
- Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. jun 2015. URL https://arxiv.org/abs/1506.04214.
- Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/e5f6ad6ce374177eef023bf5d0c018b6-Paper.pdf.
- Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2019-June, pages 9146–9154, 2019. ISBN 9781728132938. doi: 10.1109/CVPR.2019.00937.
- Vincent Le Guen and Nicolas Thome. Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020b. doi: 10.1109/CVPR42600.2020.01149.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
- Generative Adversarial Nets. In Z Ghahramani, M Welling, C Cortes, N Lawrence, and K Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 27, 2014. URL https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf.
- Stochastic Adversarial Video Prediction. apr 2018. URL http://arxiv.org/abs/1804.01523.
- Skilful precipitation nowcasting using deep generative models of radar. Nature, 597(7878):672–677, 2021.
- VideoGPT: Video Generation using VQ-VAE and Transformers. 2021. URL http://arxiv.org/abs/2104.10157.
- Unsupervised learning of video representations using lstms. In International conference on machine learning, pages 843–852. PMLR, 2015.
- Self-supervised visual planning with temporal skip connections, 2017.
- Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012.
- Sky image forecasting with generative adversarial networks for cloud coverage prediction. In 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), pages 1–7. IEEE, 2019.
- Precise forecasting of sky images using spatial warping. In In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1136–1144, 2021.
- Neural discrete representation learning. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/7a98af17e63a0ac09ce2e96d03992fbc-Paper.pdf.
- Generative pretraining from pixels. In International conference on machine learning, pages 1691–1703. PMLR, 2020.
- Age estimation via face images: a survey. EURASIP Journal on Image and Video Processing, 2018(1):1–35, 2018.
- U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
- Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
- Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
- Probabilistic forecasting. Annual Review of Statistics and Its Application, 1:125–151, 2014.
- Robert L Winkler. A decision-theoretic approach to interval estimation. Journal of the American Statistical Association, 67(337):187–191, 1972.
- Aldo da Rosa. Fundamentals of Renewable Energy Processes. 2009. ISBN 9780123746399. doi: 10.1016/B978-0-12-374639-9.X0001-2.
- Skipp’d: a sky images and photovoltaic power generation dataset for short-term solar forecasting. arXiv preprint arXiv:2207.00913, 2022a.
- Estimating clear-sky pv electricity production without exogenous data. Solar Energy Advances, 2:100015, 2022.
- Open-source ground-based sky image datasets for very short-term solar forecasting, cloud analysis and modeling: A comprehensive survey. arXiv preprint arXiv:2211.14709, 2022b.
- An impartial take to the cnn vs transformer robustness contest. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIII, pages 466–480. Springer, 2022.
- Khawar Islam. Recent advances in vision transformer: A survey and outlook of recent work. arXiv preprint arXiv:2203.01536, 2022.
- A battle of network structures: An empirical study of cnn, transformer, and mlp. arXiv preprint arXiv:2108.13002, 2021.
- An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- Sky-image-based solar forecasting using deep learning with multi-location data: training models locally, globally or via transfer learning? arXiv preprint arXiv:2211.02108, 2022c.