Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Model-Free Load Frequency Control of Nonlinear Power Systems Based on Deep Reinforcement Learning (2403.04374v1)

Published 7 Mar 2024 in eess.SY, cs.AI, and cs.SY

Abstract: Load frequency control (LFC) is widely employed in power systems to stabilize frequency fluctuation and guarantee power quality. However, most existing LFC methods rely on accurate power system modeling and usually ignore the nonlinear characteristics of the system, limiting controllers' performance. To solve these problems, this paper proposes a model-free LFC method for nonlinear power systems based on deep deterministic policy gradient (DDPG) framework. The proposed method establishes an emulator network to emulate power system dynamics. After defining the action-value function, the emulator network is applied for control actions evaluation instead of the critic network. Then the actor network controller is effectively optimized by estimating the policy gradient based on zeroth-order optimization (ZOO) and backpropagation algorithm. Simulation results and corresponding comparisons demonstrate the designed controller can generate appropriate control actions and has strong adaptability for nonlinear power systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. X.-C. Shangguan, C.-K. Zhang, Y. He, L. Jin, L. Jiang, J. W. Spencer, and M. Wu, “Robust load frequency control for power system considering transmission delay and sampling period,” IEEE Transactions on Industrial Informatics, vol. 17, no. 8, pp. 5292–5303, 2021.
  2. K. Liao and Y. Xu, “A robust load frequency control scheme for power systems based on second-order sliding mode and extended disturbance observer,” IEEE Transactions on Industrial Informatics, vol. 14, no. 7, pp. 3076–3086, 2018.
  3. X. Bu, W. Yu, L. Cui, Z. Hou, and Z. Chen, “Event-triggered data-driven load frequency control for multiarea power systems,” IEEE Transactions on Industrial Informatics, vol. 18, no. 9, pp. 5982–5991, 2022.
  4. L. Cai, Z. He, and H. Hu, “A new load frequency control method of multi-area power system via the viewpoints of port-hamiltonian system and cascade system,” IEEE Transactions on Power Systems, vol. 32, no. 3, pp. 1689–1700, 2016.
  5. N. Chuang, “Robust load-frequency control in interconnected power systems,” IET control theory & Applications, vol. 10, no. 1, pp. 67–75, 2016.
  6. X. Su, X. Liu, and Y.-D. Song, “Event-triggered sliding-mode control for multi-area power systems,” IEEE Transactions on Industrial Electronics, vol. 64, no. 8, pp. 6732–6741, 2017.
  7. M. K. AL-Nussairi, S. D. Al-Majidi, A. R. Hussein, and R. Bayindir, “Design of a load frequency control based on a fuzzy logic for single area networks,” in 2021 10th International Conference on Renewable Energy Research and Application (ICRERA), pp. 216–220, IEEE, 2021.
  8. M. Siti, N. Mbungu, D. Tungadio, B. Banza, and L. Ngoma, “Application of load frequency control method to a multi-microgrid with energy storage system,” Journal of Energy Storage, vol. 52, p. 104629, 2022.
  9. M. Zhang, S. Dong, Z.-G. Wu, G. Chen, and X. Guan, “Reliable event-triggered load frequency control of uncertain multiarea power systems with actuator failures,” IEEE Transactions on Automation Science and Engineering, vol. 20, no. 4, pp. 1545–5955, 2023.
  10. Z. Wu, H. Mo, J. Xiong, and M. Xie, “Adaptive event-triggered observer-based output feedback ℒ∞subscriptℒ\mathcal{L}_{\infty}caligraphic_L start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT load frequency control for networked power systems,” IEEE Transactions on Industrial Informatics, vol. 16, no. 6, pp. 3952–3962, 2019.
  11. M. Zhang, S. Dong, P. Shi, G. Chen, and X. Guan, “Distributed observer-based event-triggered load frequency control of multiarea power systems under cyber attacks,” IEEE Transactions on Automation Science and Engineering, vol. 20, no. 4, pp. 2435–2444, 2023.
  12. G. Liu, J. H. Park, C. Hua, and Y. Li, “Hybrid dynamic event-triggered load frequency control for power systems with unreliable transmission networks,” IEEE Transactions on Cybernetics, doi: 10.1109/TCY-B.2022.3163271.
  13. M. Zhang, Z. Wu, J. Yan, R. Lu, and X. Guan, “Attack-resilient optimal pmu placement via reinforcement learning guided tree search in smart grids,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 1919–1929, 2022.
  14. V. P. Singh, N. Kishor, and P. Samuel, “Distributed multi-agent system-based load frequency control for multi-area power system in smart grid,” IEEE Transactions on Industrial Electronics, vol. 64, no. 6, pp. 5151–5160, 2017.
  15. L. Yin, T. Yu, L. Zhou, L. Huang, X. Zhang, and B. Zheng, “Artificial emotional reinforcement learning for automatic generation control of large-scale interconnected power grids,” IET Generation, Transmission & Distribution, vol. 11, no. 9, pp. 2305–2313, 2017.
  16. Z. Yan and Y. Xu, “Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with continuous action search,” IEEE Transactions on Power Systems, vol. 34, no. 2, pp. 1653–1656, 2018.
  17. L. Xi, L. Zhou, Y. Xu, and X. Chen, “A multi-step unified reinforcement learning method for automatic generation control in multi-area interconnected power grid,” IEEE Transactions on Sustainable Energy, vol. 12, no. 2, pp. 1406–1415, 2020.
  18. S. Rozada, D. Apostolopoulou, and E. Alonso, “Load frequency control: A deep multi-agent reinforcement learning approach,” in 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5, IEEE, 2020.
  19. J. Li, T. Yu, and X. Zhang, “Coordinated load frequency control of multi-area integrated energy system using multi-agent deep reinforcement learning,” Applied Energy, vol. 306, p. 117900, 2022.
  20. Z. Yan and Y. Xu, “A multi-agent deep reinforcement learning method for cooperative load frequency control of a multi-area power system,” IEEE Transactions on Power Systems, vol. 35, no. 6, pp. 4599–4608, 2020.
  21. F. Yang, D. Huang, D. Li, S. Lin, S. Muyeen, and H. Zhai, “Data-driven load frequency control based on multi-agent reinforcement learning with attention mechanism,” IEEE Transactions on Power Systems, 2022.
  22. T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015.
  23. P.-Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh, “Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,” in Proceedings of the 10th ACM workshop on artificial intelligence and security, 2017, pp. 15–26.
  24. F. Daneshfar and H. Bevrani, “Multiobjective design of load frequency control using genetic algorithms,” International Journal of Electrical Power & Energy Systems, vol. 42, no. 1, pp. 257–263, 2012.
Citations (7)

Summary

We haven't generated a summary for this paper yet.