Andes_gym: A Versatile Environment for Deep Reinforcement Learning in Power Systems

Abstract: This paper presents Andes_gym, a versatile and high-performance reinforcement learning environment for power system studies. The environment leverages the modeling and simulation capability of ANDES and the reinforcement learning (RL) environment OpenAI Gym to enable the prototyping and demonstration of RL algorithms for power systems. The architecture of the proposed software tool is elaborated to provide the observation and action interfaces for RL algorithms. An example is shown to rapidly prototype a load-frequency control algorithm based on RL trained by available algorithms. The proposed environment is highly generalized by supporting all the power system dynamic models available in ANDES and numerous RL algorithms available for OpenAI Gym.