Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dynamic Energy Dispatch Based on Deep Reinforcement Learning in IoT-Driven Smart Isolated Microgrids (2002.02581v2)

Published 7 Feb 2020 in cs.LG, eess.SP, and stat.ML

Abstract: Microgrids (MGs) are small, local power grids that can operate independently from the larger utility grid. Combined with the Internet of Things (IoT), a smart MG can leverage the sensory data and machine learning techniques for intelligent energy management. This paper focuses on deep reinforcement learning (DRL)-based energy dispatch for IoT-driven smart isolated MGs with diesel generators (DGs), photovoltaic (PV) panels, and a battery. A finite-horizon Partial Observable Markov Decision Process (POMDP) model is formulated and solved by learning from historical data to capture the uncertainty in future electricity consumption and renewable power generation. In order to deal with the instability problem of DRL algorithms and unique characteristics of finite-horizon models, two novel DRL algorithms, namely, finite-horizon deep deterministic policy gradient (FH-DDPG) and finite-horizon recurrent deterministic policy gradient (FH-RDPG), are proposed to derive energy dispatch policies with and without fully observable state information. A case study using real isolated MG data is performed, where the performance of the proposed algorithms are compared with the other baseline DRL and non-DRL algorithms. Moreover, the impact of uncertainties on MG performance is decoupled into two levels and evaluated respectively.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. W. Li, T. Logenthiran, V.-T. Phan, and W. L. Woo, “Implemented iot-based self-learning home management system (shms) for singapore,” IEEE Internet of Things Journal, vol. 5, no. 3, pp. 2212–2219, 2018.
  2. S. Li, J. Yang, W. Song, and A. Chen, “A real-time electricity scheduling for residential home energy management,” IEEE Internet of Things Journal, vol. 6, no. 2, pp. 2602–2611, 2018.
  3. G. Bedi, G. K. Venayagamoorthy, R. Singh, R. R. Brooks, and K.-C. Wang, “Review of internet of things (iot) in electric power and energy systems,” IEEE Internet of Things Journal, vol. 5, no. 2, pp. 847–870, 2018.
  4. D. Minoli, K. Sohraby, and B. Occhiogrosso, “Iot considerations, requirements, and architectures for smart buildings—energy optimization and next-generation building management systems,” IEEE Internet of Things Journal, vol. 4, no. 1, pp. 269–283, 2017.
  5. L. Lei, Y. Tan, S. Liu, K. Zheng, and X. Shen, “Deep reinforcement learning for autonomous internet of things: Model, applications and challenges,” arXiv preprint arXiv:1907.09059, 2019.
  6. W. Su, J. Wang, and J. Roh, “Stochastic energy scheduling in microgrids with intermittent renewable energy resources,” IEEE Transactions on Smart Grid, vol. 5, no. 4, pp. 1876–1883, Jul. 2014.
  7. Z. Wang, B. Chen, J. Wang, M. M. Begovic, and C. Chen, “Coordinated energy management of networked microgrids in distribution systems,” IEEE Transactions on Smart Grid, vol. 6, no. 1, pp. 45–53, Jan. 2015.
  8. H. Farzin, M. Fotuhi-Firuzabad, and M. Moeini-Aghtaie, “A stochastic multi-objective framework for optimal scheduling of energy storage systems in microgrids,” IEEE Transactions on Smart Grid, vol. 8, no. 1, pp. 117–127, Jan. 2017.
  9. Y. Zhang, N. Gatsis, and G. B. Giannakis, “Robust energy management for microgrids with high-penetration renewables,” IEEE Transactions on Sustainable Energy, vol. 4, no. 4, pp. 944–953, Oct. 2013.
  10. J. S. Giraldo, J. A. Castrillon, J. C. López, M. J. Rider, and C. A. Castro, “Microgrids energy management using robust convex programming,” IEEE Transactions on Smart Grid, vol. 10, no. 4, pp. 4520–4530, Jul. 2019.
  11. D. E. Olivares, C. A. Cañizares, and M. Kazerani, “A centralized energy management system for isolated microgrids,” IEEE Transactions on Smart Grid, vol. 5, no. 4, pp. 1864–1875, Jul. 2014.
  12. D. Zhu and G. Hug, “Decomposed stochastic model predictive control for optimal dispatch of storage and generation,” IEEE Transactions on Smart Grid, vol. 5, no. 4, pp. 2044–2053, Jul. 2014.
  13. J. Sachs and O. Sawodny, “A two-stage model predictive control strategy for economic diesel-PV-battery island microgrid operation in rural areas,” IEEE Transactions on Sustainable Energy, vol. 7, no. 3, pp. 903–913, Jul. 2016.
  14. J. D. Lara, D. E. Olivares, and C. A. Cañizares, “Robust energy management of isolated microgrids,” IEEE Systems Journal, vol. 13, no. 1, pp. 680–691, Mar. 2019.
  15. P. Zeng, H. Li, H. He, and S. Li, “Dynamic energy management of a microgrid using approximate dynamic programming and deep recurrent neural network learning,” IEEE Transactions on Smart Grid, 2018.
  16. H. Shuai, J. Fang, X. Ai, J. Wen, and H. He, “Optimal real-time operation strategy for microgrid: An adp-based stochastic nonlinear optimization approach,” IEEE Transactions on Sustainable Energy, vol. 10, no. 2, pp. 931–942, 2018.
  17. H. Shuai, J. Fang, X. Ai, Y. Tang, J. Wen, and H. He, “Stochastic optimization of economic dispatch for microgrid based on approximate dynamic programming,” IEEE Transactions on Smart Grid, vol. 10, no. 3, pp. 2440–2452, May 2019.
  18. W. Shi, N. Li, C. Chu, and R. Gadh, “Real-time energy management in microgrids,” IEEE Transactions on Smart Grid, vol. 8, no. 1, pp. 228–238, Jan. 2017.
  19. W. Hu, P. Wang, and H. B. Gooi, “Toward optimal energy management of microgrids via robust two-stage optimization,” IEEE Transactions on Smart Grid, vol. 9, no. 2, pp. 1161–1174, Mar. 2018.
  20. V. François-Lavet, D. Taralla, D. Ernst, and R. Fonteneau, “Deep reinforcement learning solutions for energy microgrids management,” in European Workshop on Reinforcement Learning (EWRL 2016), 2016.
  21. Y. Ji, J. Wang, J. Xu, X. Fang, and H. Zhang, “Real-time energy management of a microgrid using deep reinforcement learning,” Energies, vol. 12, p. 2291, 2019.
  22. G. K. Venayagamoorthy, R. K. Sharma, P. K. Gautam, and A. Ahmadi, “Dynamic energy management system for a smart microgrid,” IEEE transactions on neural networks and learning systems, vol. 27, no. 8, pp. 1643–1656, 2016.
  23. E. Foruzan, L. Soh, and S. Asgarpoor, “Reinforcement learning approach for optimal distributed energy management in a microgrid,” IEEE Transactions on Power Systems, vol. 33, no. 5, pp. 5749–5758, Sep. 2018.
  24. L. Yu, W. Xie, D. Xie, Y. Zou, D. Zhang, Z. Sun, L. Zhang, Y. Zhang, and T. Jiang, “Deep reinforcement learning for smart home energy management,” IEEE Internet of Things Journal, 2019.
  25. M. S. Munir, S. F. Abedin, N. H. Tran, and C. S. Hong, “When edge computing meets microgrid: A deep reinforcement learning approach,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 7360–7374, 2019.
  26. W. B. Powell and S. Meisel, “Tutorial on stochastic optimization in energy—part i: Modeling and policies,” IEEE Transactions on Power Systems, vol. 31, no. 2, pp. 1459–1467, 2016.
  27. ——, “Tutorial on stochastic optimization in energy—part ii: An energy storage illustration,” IEEE Transactions on Power Systems, vol. 31, no. 2, pp. 1468–1475, 2016.
  28. Q. Zheng, K. Zheng, H. Zhang, and V. C. M. Leung, “Delay-optimal virtualized radio resource scheduling in software-defined vehicular networks via stochastic learning,” IEEE Transactions on Vehicular Technology, vol. 65, no. 10, pp. 7857–7867, Oct. 2016.
  29. B. O’Donoghue, I. Osband, R. Munos, and V. Mnih, “The uncertainty bellman equation and exploration.”
  30. N. Heess, G. Wayne, D. Silver, T. Lillicrap, T. Erez, and Y. Tassa, “Learning continuous control policies by stochastic value gradients,” in Advances in Neural Information Processing Systems, 2015, pp. 2944–2952.
  31. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529, 2015.
  32. H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-learning,” in Thirtieth AAAI Conference on Artificial Intelligence, 2016.
  33. R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning, vol. 8, pp. 229–256, 1992.
  34. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
  35. V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning, 2016, pp. 1928–1937.
  36. J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International Conference on Machine Learning, 2015, pp. 1889–1897.
  37. N. Heess, J. J. Hunt, T. P. Lillicrap, and D. Silver, “Memory-based control with recurrent neural networks,” arXiv preprint arXiv:1512.04455, 2015.
  38. A. Irpan, “Deep reinforcement learning doesn’t work yet,” https://www.alexirpan.com/2018/02/14/rl-hard.html, 2018.
  39. Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep reinforcement learning for continuous control.”
  40. C. Qiu, Y. Hu, Y. Chen, and B. Zeng, “Deep deterministic policy gradient (ddpg)-based energy harvesting wireless communications,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 8577–8588, 2019.
  41. I. Grondman, H. Xu, S. Jagannathan, and R. Babuška, “Solutions to finite horizon cost problems using actor-critic reinforcement learning,” in The 2013 International Joint Conference on Neural Networks (IJCNN).   IEEE, 2013, pp. 1–7.
  42. H. Bergveld, “Battery management systems design by modelling, university twente,” Ph.D. dissertation, Ph. D. thesis, 2001.
  43. M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., “Tensorflow: A system for large-scale machine learning,” in 12th {normal-{\{{USENIX}normal-}\}} symposium on operating systems design and implementation ({normal-{\{{OSDI}normal-}\}} 16), 2016, pp. 265–283.
  44. E. Todorov and W. Li, “A generalized iterative lqg method for locally-optimal feedback control of constrained nonlinear stochastic systems,” in Proceedings of the 2005, American Control Conference, 2005.   IEEE, 2005, pp. 300–306.
  45. M. Djurovic, A. Milacic, and M. Krsulja, “A simplified model of quadratic cost function for thermal generators,” in 23rd International DAAAM Symposium Intelligent Manufacturing & Automation: Focus on Sustainability, 2012.
Citations (63)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now