Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SkyGPT: Probabilistic Short-term Solar Forecasting Using Synthetic Sky Videos from Physics-constrained VideoGPT (2306.11682v1)

Published 20 Jun 2023 in cs.CV

Abstract: In recent years, deep learning-based solar forecasting using all-sky images has emerged as a promising approach for alleviating uncertainty in PV power generation. However, the stochastic nature of cloud movement remains a major challenge for accurate and reliable solar forecasting. With the recent advances in generative artificial intelligence, the synthesis of visually plausible yet diversified sky videos has potential for aiding in forecasts. In this study, we introduce \emph{SkyGPT}, a physics-informed stochastic video prediction model that is able to generate multiple possible future images of the sky with diverse cloud motion patterns, by using past sky image sequences as input. Extensive experiments and comparison with benchmark video prediction models demonstrate the effectiveness of the proposed model in capturing cloud dynamics and generating future sky images with high realism and diversity. Furthermore, we feed the generated future sky images from the video prediction models for 15-minute-ahead probabilistic solar forecasting for a 30-kW roof-top PV system, and compare it with an end-to-end deep learning baseline model SUNSET and a smart persistence model. Better PV output prediction reliability and sharpness is observed by using the predicted sky images generated with SkyGPT compared with other benchmark models, achieving a continuous ranked probability score (CRPS) of 2.81 (13\% better than SUNSET and 23\% better than smart persistence) and a Winkler score of 26.70 for the test set. Although an arbitrary number of futures can be generated from a historical sky image sequence, the results suggest that 10 future scenarios is a good choice that balances probabilistic solar forecasting performance and computational cost.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. IPCC. IPCC Special Report on Renewable Energy Sources and Climate Change Mitigation (Chapter 9). Prepared by Working Group III of the Intergovernmental Panel on Climate Change. Cambridge University Press, 2011. URL http://www.ipcc-wg3.de/srren-report/.
  2. Yuchi Sun. Short-term Solar Forecast Using Convolutional Neural Networks with Sky Images. PhD thesis, 2019. URL https://www.proquest.com/dissertations-theses/short-term-solar-forecast-using-convolutional/docview/2466057410/se-2.
  3. Review on probabilistic forecasting of photovoltaic power production and electricity consumption. Renewable and Sustainable Energy Reviews, 81:1484–1512, 2018.
  4. Intra-hour forecasting with a total sky imager at the uc san diego solar energy testbed. Solar Energy, 85(11):2881–2893, 2011.
  5. Intra-hour dni forecasting based on cloud tracking image analysis. Solar Energy, 91:327–336, 2013.
  6. Cloud-tracking methodology for intra-hour DNI forecasting. Solar Energy, 102:267–275, April 2014. ISSN 0038-092X. doi: 10.1016/j.solener.2014.01.030.
  7. Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning. Solar Energy, 98:592–603, December 2013. ISSN 0038-092X. doi: 10.1016/j.solener.2013.10.020.
  8. Real-time prediction intervals for intra-hour DNI forecasts. Renewable Energy, 83:234–244, 2015a. ISSN 18790682. doi: 10.1016/j.renene.2015.04.022. URL http://dx.doi.org/10.1016/j.renene.2015.04.022.
  9. Short-term reforecasting of power output from a 48 MWe solar PV plant. Solar Energy, 112:68–77, feb 2015b. ISSN 0038-092X. doi: 10.1016/J.SOLENER.2014.11.017. URL https://www.sciencedirect.com/science/article/pii/S0038092X14005611.
  10. Adaptive image features for intra-hour solar forecasts. Journal of Renewable and Sustainable Energy, 11(3):036101, May 2019. doi: 10.1063/1.5091952.
  11. Short-term solar power forecast with deep learning: Exploring optimal input and output configuration. Solar Energy, 188:730–741, aug 2019. doi: 10.1016/j.solener.2019.06.041. URL https://linkinghub.elsevier.com/retrieve/pii/S0038092X19306164.
  12. PV power output prediction from sky images using convolutional neural network: The comparison of sky-condition-specific sub-models and an end-to-end model. Journal of Renewable and Sustainable Energy, 12(4):046101, jul 2020. doi: 10.1063/5.0014016. URL http://aip.scitation.org/doi/10.1063/5.0014016.
  13. SolarNet: A sky image-based deep convolutional neural network for intra-hour solar forecasting. Solar Energy, 204(April):71–78, 2020. doi: 10.1016/j.solener.2020.03.083. URL https://doi.org/10.1016/j.solener.2020.03.083.
  14. Convolutional Neural Networks Applied to Sky Images for Short-Term Solar Irradiance Forecasting. In EU PVSEC, pages 1834 – 1837, 2020. ISBN 3-936338-73-6. doi: 10.4229/EUPVSEC20202020-6BV.5.15. URL https://www.eupvsec-proceedings.com/proceedings?paper=49346.
  15. Resampling and data augmentation for short-term pv output prediction based on an imbalanced sky images dataset using convolutional neural networks. Solar Energy, 224:341–354, 2021.
  16. Convolutional neural networks for intra-hour solar forecasting based on sky image sequences. Applied Energy, 310:118438, 2022.
  17. Deep photovoltaic nowcasting. Solar Energy, 176(September):267–276, 2018. doi: 10.1016/j.solener.2018.10.024. URL https://doi.org/10.1016/j.solener.2018.10.024.
  18. Benchmarking of deep learning irradiance forecasting models from sky images – An in-depth analysis. Solar Energy, 224:855–867, August 2021. ISSN 0038-092X. doi: 10.1016/j.solener.2021.05.056. URL https://linkinghub.elsevier.com/retrieve/pii/S0038092X21004266.
  19. ECLIPSE: Envisioning CLoud Induced Perturbations in Solar Energy. Applied Energy, 326:119924, November 2022. ISSN 0306-2619. doi: 10.1016/j.apenergy.2022.119924.
  20. Vincent Le Guen and Nicolas Thome. A deep physical model for solar irradiance forecasting with fisheye images. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2020-June:2685–2688, 2020a. ISSN 21607516. doi: 10.1109/CVPRW50498.2020.00323.
  21. Digital particle image velocimetry. Experiments in fluids, 10(4):181–193, 1991.
  22. The computation of optical flow. ACM computing surveys (CSUR), 27(3):433–466, 1995.
  23. An ensemble prediction intervals approach for short-term pv power forecasting. Solar Energy, 155:1072–1083, 2017.
  24. Short-term solar irradiance forecasting using calibrated probabilistic models. NeurIPS 2020 Workshop on Tackling Climate Change with Machine Learning, 2020.
  25. Omnivision forecasting: Combining satellite and sky images for improved deterministic and probabilistic intra-hour solar energy predictions. Applied Energy, 336:120818, April 2023. ISSN 0306-2619. doi: 10.1016/j.apenergy.2023.120818.
  26. A Review on Deep Learning Techniques for Video Prediction. apr 2020. URL http://arxiv.org/abs/2004.05214.
  27. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  28. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. jun 2015. URL https://arxiv.org/abs/1506.04214.
  29. Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/e5f6ad6ce374177eef023bf5d0c018b6-Paper.pdf.
  30. Memory in memory: A predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2019-June, pages 9146–9154, 2019. ISBN 9781728132938. doi: 10.1109/CVPR.2019.00937.
  31. Vincent Le Guen and Nicolas Thome. Disentangling physical dynamics from unknown factors for unsupervised video prediction. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020b. doi: 10.1109/CVPR42600.2020.01149.
  32. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  33. Generative Adversarial Nets. In Z Ghahramani, M Welling, C Cortes, N Lawrence, and K Q Weinberger, editors, Advances in Neural Information Processing Systems, volume 27, 2014. URL https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf.
  34. Stochastic Adversarial Video Prediction. apr 2018. URL http://arxiv.org/abs/1804.01523.
  35. Skilful precipitation nowcasting using deep generative models of radar. Nature, 597(7878):672–677, 2021.
  36. VideoGPT: Video Generation using VQ-VAE and Transformers. 2021. URL http://arxiv.org/abs/2104.10157.
  37. Unsupervised learning of video representations using lstms. In International conference on machine learning, pages 843–852. PMLR, 2015.
  38. Self-supervised visual planning with temporal skip connections, 2017.
  39. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012.
  40. Sky image forecasting with generative adversarial networks for cloud coverage prediction. In 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), pages 1–7. IEEE, 2019.
  41. Precise forecasting of sky images using spatial warping. In In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1136–1144, 2021.
  42. Neural discrete representation learning. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/7a98af17e63a0ac09ce2e96d03992fbc-Paper.pdf.
  43. Generative pretraining from pixels. In International conference on machine learning, pages 1691–1703. PMLR, 2020.
  44. Age estimation via face images: a survey. EURASIP Journal on Image and Video Processing, 2018(1):1–35, 2018.
  45. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
  46. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  47. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  48. Probabilistic forecasting. Annual Review of Statistics and Its Application, 1:125–151, 2014.
  49. Robert L Winkler. A decision-theoretic approach to interval estimation. Journal of the American Statistical Association, 67(337):187–191, 1972.
  50. Aldo da Rosa. Fundamentals of Renewable Energy Processes. 2009. ISBN 9780123746399. doi: 10.1016/B978-0-12-374639-9.X0001-2.
  51. Skipp’d: a sky images and photovoltaic power generation dataset for short-term solar forecasting. arXiv preprint arXiv:2207.00913, 2022a.
  52. Estimating clear-sky pv electricity production without exogenous data. Solar Energy Advances, 2:100015, 2022.
  53. Open-source ground-based sky image datasets for very short-term solar forecasting, cloud analysis and modeling: A comprehensive survey. arXiv preprint arXiv:2211.14709, 2022b.
  54. An impartial take to the cnn vs transformer robustness contest. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIII, pages 466–480. Springer, 2022.
  55. Khawar Islam. Recent advances in vision transformer: A survey and outlook of recent work. arXiv preprint arXiv:2203.01536, 2022.
  56. A battle of network structures: An empirical study of cnn, transformer, and mlp. arXiv preprint arXiv:2108.13002, 2021.
  57. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  58. Sky-image-based solar forecasting using deep learning with multi-location data: training models locally, globally or via transfer learning? arXiv preprint arXiv:2211.02108, 2022c.
Citations (2)

Summary

  • The paper introduces SkyGPT, a two-stage framework that integrates VideoGPT with physics-based constraints to generate realistic cloud dynamics.
  • It leverages a vector quantized variational auto-encoder and a conditional transformer to produce diverse, plausible future sky scenarios.
  • SkyGPT outperforms benchmarks like SUNSET in metrics such as CRPS and Winkler Score, leading to more reliable short-term solar forecasting.

Probabilistic Short-term Solar Forecasting with SkyGPT

The research paper titled "SkyGPT: Probabilistic Short-term Solar Forecasting Using Synthetic Sky Videos from Physics-constrained VideoGPT" presents an innovative advancement in short-term solar power prediction through the development of the SkyGPT forecasting system. This two-stage deep learning framework leverages generative models to predict cloud dynamics with particular attention towards generating plausible sky videos, providing a novel approach to photovoltaic (PV) power forecasting.

Key Contributions and Methodology

The primary contribution of this work is the development of SkyGPT, which combines the strengths of VideoGPT with the inclusion of physics-informed constraints from the PhyDNet model architecture. This setup allows for generating diverse and realistic future sky videos by modeling cloud dynamics effectively using past sky image sequences. SkyGPT utilizes a vector quantized variational auto-encoder (VQ-VAE) to compress high dimensional video data, and a conditional transformer is used to predict future tokens, facilitating the generation of numerous possible future scenarios. The incorporation of physical laws via PhyCell in the transformer allows it to retain the dynamics of cloud motion which is crucial for accurate forecasting.

In the subsequent stage, these predicted sky images are input into a subsequent convolutional neural network (CNN)-based framework, specifically a modified U-Net architecture, to estimate future PV output. This setup contrasts with traditional deterministic models by offering a probabilistic forecasting approach that attempts to capture uncertainty through the stochastic generation of cloud scenarios.

Results and Performance

Experimental evaluations, as discussed, demonstrate the effectiveness of SkyGPT in generating realistic sky imagery and enhancing solar forecast accuracy. When benchmarked against existing methods such as SUNSET and smart persistence models, the proposed methodology displayed improved performance in metrics such as Continuous Ranked Probability Score (CRPS) and Winkler Score (WS).

Particularly, the stochastic nature of SkyGPT shines through in cases showing highly variable weather conditions, where deterministic models typically fail to accurately track sudden changes in cloud cover. The generation of multiple future scenarios in SkyGPT contributes to superior probabilistic prediction by covering a wider range of potential future states, resulting in increased reliability and sharper predictions.

Implications and Future Directions

The implications of this research are manifold, both for practical applications and theoretical advancements in AI-driven forecasting. Practically, improved short-term solar forecast accuracy can aid in the stability and operational efficiency of power grids heavily reliant on solar sources, reducing reliance on supplementary dispatchable resources. Theoretically, the integrations of physics-informed constraints into generative models open new pathways in reliably modeling stochastic processes in environmental settings, suggesting similar approaches may be beneficial across various domains involving fluid dynamics or other meteorological phenomena.

Future explorations could emphasize refining the physics-constrained components for enhanced model competence in complex situations. Investigating the integration of additional meteorological data sources and improved transfer learning techniques to accommodate varying environmental conditions across geographic regions is another promising direction to amplify this framework’s generalizability. Furthermore, enhancing the PV output prediction model sophistication could advance the frontiers of accuracy, particularly through leveraging novel architectures like Vision Transformers.

In conclusion, SkyGPT presents significant progress in probabilistic solar forecasting, showcasing the potential of generative AI techniques when augmented with domain-specific knowledge constraints. This paper exemplifies the convergence of AI and physical science, reinforcing the notion that collaborative integration can spearhead cutting-edge developments in forecasting technologies.