Adam-based Augmented Random Search for Control Policies for Distributed Energy Resource Cyber Attack Mitigation

Abstract: Volt-VAR and Volt-Watt control functions are mechanisms that are included in distributed energy resource (DER) power electronic inverters to mitigate excessively high or low voltages in distribution systems. In the event that a subset of DER have had their Volt-VAR and Volt-Watt settings compromised as part of a cyber-attack, we propose a mechanism to control the remaining set of non-compromised DER to ameliorate large oscillations in system voltages and large voltage imbalances in real time. To do so, we construct control policies for individual non-compromised DER, directly searching the policy space using an Adam-based augmented random search (ARS). In this paper we show that, compared to previous efforts aimed at training policies for DER cybersecurity using deep reinforcement learning (DRL), the proposed approach is able to learn optimal (and sometimes linear) policies an order of magnitude faster than conventional DRL techniques (e.g., Proximal Policy Optimization).