Leading-Power Safety in ML-Driven Power Systems

• Leading-Power Safety is a framework that guarantees machine-learning controllers comply with operational limits, maintain stability, and manage risks in power systems.
• It integrates safety-by-design with runtime monitoring by combining model predictive control, safety shields, and adversarial robustness to counter disturbances and data imperfections.
• Empirical results indicate that explicit safety measures and fallback controls significantly reduce constraint violations and improve performance under adversarial conditions.

Searching arXiv for the specified power-systems safety papers to ground the article and citations. Search query: (Chen et al., 2021) Leading-Power Safety denotes safety-by-design and safety-in-operation for machine-learning and reinforcement-learning-driven power systems. In this usage, a learned controller is safe when its action sequence keeps the system within operational limits, achieves stability, and maintains acceptable risk levels even under disturbances and data imperfections; at runtime, safety additionally requires monitoring, detection, and mitigation of anomalies or adversarial data, together with robust fallback control (Chen et al., 2021). Recent work develops this concept along three complementary lines: adversarial robustness analysis for voltage regulation and topology control (Chen et al., 2021), model-based run-time assurance with predictive safety filters and automatically synthesized model predictive control baselines (Eichelbeck et al., 2024), and safe reinforcement learning for voltage stability emergency control using a learned Dynamic Action-Joined Security Margin and gradient projection correction (Bi et al., 2024).

1. Safety semantics in learning-based power operations

In learning-based power system operations, safety-by-design refers to design-time guarantees that a learned controller respects power system physics and operational constraints across expected operating conditions and contingencies. Safety-in-operation refers to runtime assurance that actions will not drive the grid into unsafe regions, with monitoring, detection, and mitigation of anomalies or adversarial data, and robust fallback control (Chen et al., 2021). This distinction is central because many RL formulations optimize reward but do not explicitly encode hard constraints.

The operational content of safety is multi-layered. Hard constraint satisfaction requires voltage magnitude bounds $V_i \in [V_{\min}, V_{\max}]$, thermal ratings such as $|S_\ell| \leq S_\ell^{\max}$, feasible AC power flow solutions, and topology feasibility including radiality constraints in distribution systems and valid bus-branch configurations in transmission systems. Stability constraints extend this notion to frequency and rotor-angle stability following disturbances, bounded rate of change of frequency and frequency nadir, and Lyapunov-style contraction conditions of the form $V(s') - V(s) \leq -W(s)$. Risk-bounded formulations add probabilistic safety, for example $P(g_k(s,a)\leq 0)\geq 1-\alpha$, and tail-risk control through CVaR constraints. The same safety envelope is often required to satisfy $N-1$ security and resilience to malicious or worst-case bounded observation perturbations, including stealthy attacks that evade bad-data detection (Chen et al., 2021).

CommonPower adopts a narrower but explicit case-study definition: safety is guaranteed satisfaction of operational constraints at all times, during training and deployment, including voltage and thermal line limits and power balance, together with state and input bounds for controllable assets such as battery state of charge and charge/discharge power (Eichelbeck et al., 2024). Frequency and stability are outside the scope of that case study, although the library supports symbolic constraints so stability-related constraints or dynamics can be encoded if desired. By contrast, the voltage-stability emergency-control framework treats short-term voltage collapse as the safety-critical event and uses a hard safety proxy: a scenario is unsafe if $\min_j v_j(t+10) \leq 0.8$ pu, which triggers a large negative reward (Bi et al., 2024).

2. Formal models and constraint formulations

A standard power-operations formulation models states $s(t)$, actions $a(t)$, exogenous inputs $v_{\mathrm{ext}}(t)$, and dynamics

$s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$

The operational problem over horizon $|S_\ell| \leq S_\ell^{\max}$0 is

$|S_\ell| \leq S_\ell^{\max}$1

subject to dynamics, equality constraints $|S_\ell| \leq S_\ell^{\max}$2, and inequality constraints $|S_\ell| \leq S_\ell^{\max}$3 (Chen et al., 2021). In AC settings, bus injections follow the usual nodal equations

$|S_\ell| \leq S_\ell^{\max}$4

$|S_\ell| \leq S_\ell^{\max}$5

with $|S_\ell| \leq S_\ell^{\max}$6, together with voltage and thermal limits.

The RL abstraction is an MDP $|S_\ell| \leq S_\ell^{\max}$7 with state $|S_\ell| \leq S_\ell^{\max}$8, action $|S_\ell| \leq S_\ell^{\max}$9 or discrete, transition kernel $V(s') - V(s) \leq -W(s)$0, and reward $V(s') - V(s) \leq -W(s)$1. Value-based RL uses

$V(s') - V(s) \leq -W(s)$2

with $V(s') - V(s) \leq -W(s)$3, while policy-based RL solves $V(s') - V(s) \leq -W(s)$4 for $V(s') - V(s) \leq -W(s)$5 (Chen et al., 2021).

Leading-Power Safety enters this formalism through constrained and risk-aware objectives. A constrained MDP can be written as

$V(s') - V(s) \leq -W(s)$6

with the associated Lagrangian relaxation

$V(s') - V(s) \leq -W(s)$7

Chance constraints and CVaR provide probabilistic and tail-risk variants, while distributionally robust RL introduces an uncertainty set $V(s') - V(s) \leq -W(s)$8 over dynamics, noise, or perturbations and solves

$V(s') - V(s) \leq -W(s)$9

(Chen et al., 2021).

CommonPower makes the safeguarding layer explicit. For a coalition $P(g_k(s,a)\leq 0)\geq 1-\alpha$0 with state $P(g_k(s,a)\leq 0)\geq 1-\alpha$1, input $P(g_k(s,a)\leq 0)\geq 1-\alpha$2, and exogenous forecast $P(g_k(s,a)\leq 0)\geq 1-\alpha$3, it formulates MPC as

$P(g_k(s,a)\leq 0)\geq 1-\alpha$4

subject to

$P(g_k(s,a)\leq 0)\geq 1-\alpha$5

input and state bounds, and coalition coupling $P(g_k(s,a)\leq 0)\geq 1-\alpha$6 (Eichelbeck et al., 2024). The default post-posed safety shield then solves

$P(g_k(s,a)\leq 0)\geq 1-\alpha$7

subject to the coalition’s MPC feasibility constraints over the prediction horizon.

The voltage-stability emergency-control framework adopts a state-wise safe RL formulation in which a learned security margin $P(g_k(s,a)\leq 0)\geq 1-\alpha$8 must exceed a threshold $P(g_k(s,a)\leq 0)\geq 1-\alpha$9 (Bi et al., 2024). The safe action set is

$N-1$0

and the correction stage seeks

$N-1$1

Rather than solving this constrained problem exactly, the paper implements gradient ascent on the learned margin,

$N-1$2

with step size

$N-1$3

where $N-1$4 is the Lipschitz constant of the margin network with respect to $N-1$5 and $N-1$6 is the action dimension.

3. Adversarial threat models and empirical failure modes

The clearest empirical demonstration of why Leading-Power Safety is necessary comes from the black-box observation attacks studied for voltage regulation and topology control (Chen et al., 2021). The attacker has no model weights and no access to the true system model $N-1$7, but can observe states and query the policy $N-1$8 to estimate gradients by finite differences. The physical environment evolves from the true state $N-1$9, while the agent receives $\min_j v_j(t+10) \leq 0.8$0 subject to $\min_j v_j(t+10) \leq 0.8$1. Two attack classes are emphasized: action distortion, which maximizes the deviation between $\min_j v_j(t+10) \leq 0.8$2 and $\min_j v_j(t+10) \leq 0.8$3, and grid manipulation, which drives the perceived trajectory toward an attacker target such as line overflow. In the reported experiments, the perturbation budget was $\min_j v_j(t+10) \leq 0.8$4 on normalized state vectors, and PGD-like perturbations were crafted with finite-difference gradient estimates.

On the 6-bus voltage-regulation task, average reward degraded from $\min_j v_j(t+10) \leq 0.8$5 to $\min_j v_j(t+10) \leq 0.8$6 for PPO and from $\min_j v_j(t+10) \leq 0.8$7 to $\min_j v_j(t+10) \leq 0.8$8 for A2C under Action Distorted attacks, whereas MPC (DC) changed only from $\min_j v_j(t+10) \leq 0.8$9 to $s(t)$0. On the IEEE 39-bus task, PPO degraded from $s(t)$1 to $s(t)$2, A2C from $s(t)$3 to $s(t)$4, and MPC (DC) from $s(t)$5 to $s(t)$6. Targeted State Manipulated attacks drove the distance from a line’s capacity to $s(t)$7 and $s(t)$8 on the 6-bus PPO and A2C controllers, versus $s(t)$9 for MPC, and to $a(t)$0 and $a(t)$1 on IEEE 39-bus, versus $a(t)$2 for MPC. In the 6-bus system, the attack targeting line 3–6 caused that line to exceed its rating more than 8 hours within a 24-hour test by fooling the agent to charge storage during peak load.

The topology-control case study exhibits the same pattern. In L2RPN, accumulated reward per episode fell from $a(t)$3 with no attack to $a(t)$4 under Action Distorted attacks, and average survival steps fell from $a(t)$5 to $a(t)$6. The interpretation given in the paper is that action-distortion attacks reduced average reward by more than 75% and drastically shortened survival time; discrete action Q-values were often near-tied, making action flips easy under small observation perturbations. Crafting adversarial perturbations took roughly $a(t)$7 s per step in these experiments, while RL training required more than $a(t)$8k interactions.

These results are significant because they show that reward penalties alone do not constitute a safety guarantee. The same paper notes that RL agents did not explicitly encode hard constraints; safety was encouraged via large penalty terms for violations and power-flow divergence, whereas the environment or the MPC baseline carried most of the feasibility structure (Chen et al., 2021).

4. Safeguarding architectures: projection, shielding, and security margins

A major line of work replaces implicit safety-through-reward with explicit runtime assurance. CommonPower is built around a full symbolic model of the system in Pyomo, including variables, parameters, and constraints, which enables automatic synthesis of both an MPC baseline and safety shields (Eichelbeck et al., 2024). The formal guarantee is model-based: at each decision step, the safety filter projects any proposed action onto the set of actions that admit a feasible trajectory over the prediction horizon under the constraints extracted from the system’s symbolic model. This ensures that the applied action is safe with respect to the modeled constraints and dynamics.

The framework supports centralized, decentralized, and CTDE control structures. In the centralized setting, one RL agent controls the whole system and the shield can enforce both household and network-level constraints such as DC power flow and line limits. In decentralized settings, CommonPower uses a two-stage scheme: each coalition or agent is first safeguarded with local subsystem constraints, and then a balancing asset coalition computes a second-stage control action that enforces global coupling constraints such as power balance and network limits. The training loop incorporates shield feedback through reward shaping,

$a(t)$9

with a common choice

$v_{\mathrm{ext}}(t)$0

Training tuples remain $v_{\mathrm{ext}}(t)$1 so that the update uses the proposed action while penalizing deviations enforced by the shield.

The emergency-control framework follows a different but related logic (Bi et al., 2024). The policy network is Soft Actor-Critic and outputs an initial emergency control action $v_{\mathrm{ext}}(t)$2, where $v_{\mathrm{ext}}(t)$3. A second network estimates the Dynamic Action-Joined Security Margin through a dueling-style decomposition

$v_{\mathrm{ext}}(t)$4

regularized so that state-wise margin and action impact are identifiable. The corrective action implementation then uses $v_{\mathrm{ext}}(t)$5 to push risky actions into the safe set until $v_{\mathrm{ext}}(t)$6.

The paper gives an informal proposition: if $v_{\mathrm{ext}}(t)$7 is continuously differentiable and $v_{\mathrm{ext}}(t)$8-Lipschitz in $v_{\mathrm{ext}}(t)$9, then the update $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$0 with $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$1 increases $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$2 monotonically and reaches $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$3 in a finite number of steps, so the corrected action lies in the safe set. This is a stronger safety statement than simple penalty shaping because the correction stage directly enforces the learned margin constraint.

5. Data-side requirements, monitoring, and standards alignment

Leading-Power Safety is not only a controller-design problem; it is also a data, monitoring, and governance problem. Training and validation data must span seasonal and diurnal load and generation patterns, include contingencies such as $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$4 outages, rare but plausible disturbances, market-driven setpoint changes, and weather-driven ramps (Chen et al., 2021). Policies trained on narrow distributions are brittle under distribution shifts such as load-mix changes and DER growth, so runtime out-of-distribution detection is part of the safety stack.

Sensor integrity is equally central. Practical issues include time synchronization errors, missing data, scale and offset errors, and correlated noise across sensors. Recommended defenses include sensor redundancy and cross-checks through state-estimation residuals and multi-sensor voting, temporal filtering and robust estimation, and cross-validation of telemetry streams such as PMU versus SCADA combined with physics-informed checks (Chen et al., 2021). Synthetic data generation and domain randomization are used to create diverse scenarios with forecast errors, topology changes, outage events, communications latency or loss, DER tripping, and deliberate adversarial corruptions. Stress testing can then measure scenario coverage, tail performance through chance constraints and CVaR costs, and worst-case degradation in reward and violation rates under bounded perturbations.

CommonPower operationalizes part of this stack through an explicit data-provider and forecasting interface (Eichelbeck et al., 2024). A forecaster implements

$s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$5

and forecasts are exposed to both MPC and RL controllers. In experiments, noise was injected into forecasts; agents trained with noisy forecasts learned more robust policies and reduced the performance gap to MPC. The same framework also supports nonlinear AC power flow constraints symbolically, although the reported case study used a DC model.

Governance and cybersecurity requirements complete the picture. The broader safety blueprint aligns ML-enabled power-system control with $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$6 reliability design, NERC CIP controls such as CIP-005, CIP-007, CIP-008, CIP-010, CIP-011, and CIP-013, and IEC 61850 requirements for secure substation automation communications, authenticated control messages, and topology actions that comply with protection schemes and interlocking (Chen et al., 2021). Auditability requires immutable logs of inputs, actions, model version, and safety-monitor decisions. Explainability requires reporting action rationales and constraint margins. Human-in-the-loop provisions include operator confirmation for high-impact actions, graceful degradation to conservative setpoints or pre-validated rule-based controls, and emergency stop logic under simultaneous anomalies across channels.

6. Empirical performance, operating regimes, and open problems

The most mature empirical results for explicit safe RL come from the voltage-stability emergency-control framework (Bi et al., 2024). On 100 unseen extreme scenarios in the IEEE 39-bus system, the proposed method achieved $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$7, $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$8 ms, $s(t+1)=f(s(t),a(t),v_{\mathrm{ext}}(t)).$9, $|S_\ell| \leq S_\ell^{\max}$00, and $|S_\ell| \leq S_\ell^{\max}$01, compared with SAC at $|S_\ell| \leq S_\ell^{\max}$02, $|S_\ell| \leq S_\ell^{\max}$03 ms, $|S_\ell| \leq S_\ell^{\max}$04, $|S_\ell| \leq S_\ell^{\max}$05, and $|S_\ell| \leq S_\ell^{\max}$06. In the direct comparison of UVLS approaches, the proposed method achieved $|S_\ell| \leq S_\ell^{\max}$07, $|S_\ell| \leq S_\ell^{\max}$08 s, $|S_\ell| \leq S_\ell^{\max}$09, and $|S_\ell| \leq S_\ell^{\max}$10, while Typical LS had $|S_\ell| \leq S_\ell^{\max}$11, Lyapunov PG had $|S_\ell| \leq S_\ell^{\max}$12 and $|S_\ell| \leq S_\ell^{\max}$13, and CSC had $|S_\ell| \leq S_\ell^{\max}$14 and $|S_\ell| \leq S_\ell^{\max}$15. On the Guangdong Provincial Power Grid, the proposed method reported $|S_\ell| \leq S_\ell^{\max}$16, $|S_\ell| \leq S_\ell^{\max}$17 ms, and $|S_\ell| \leq S_\ell^{\max}$18, versus no correction at $|S_\ell| \leq S_\ell^{\max}$19, $|S_\ell| \leq S_\ell^{\max}$20 ms, and $|S_\ell| \leq S_\ell^{\max}$21.

The same work also reports that the dueling security-margin estimator achieved $|S_\ell| \leq S_\ell^{\max}$22, $|S_\ell| \leq S_\ell^{\max}$23, and training time $|S_\ell| \leq S_\ell^{\max}$24 s on IEEE 39-bus, compared with FCN at $|S_\ell| \leq S_\ell^{\max}$25, $|S_\ell| \leq S_\ell^{\max}$26, and training time $|S_\ell| \leq S_\ell^{\max}$27 s, and LSTM at $|S_\ell| \leq S_\ell^{\max}$28, $|S_\ell| \leq S_\ell^{\max}$29, and training time $|S_\ell| \leq S_\ell^{\max}$30 s. Its active-learning variant reduced total dataset-preparation and training time substantially while maintaining high accuracy. The paper further states that the learned Dynamic Security Region and feasible-action boundaries incorporate both reactive deficit and surplus conditions. If $|S_\ell| \leq S_\ell^{\max}$31 is abundant, most operating points lie deeper inside the admissible region and the estimator keeps actions near zero; if $|S_\ell| \leq S_\ell^{\max}$32 is scarce, correction increases shedding selectively at critical devices per $|S_\ell| \leq S_\ell^{\max}$33, preventing motor stalling and collapse. This suggests that, within that framework, Leading-Power Safety is not limited to lagging regimes but also addresses unsafe interventions in leading power-factor conditions.

CommonPower’s empirical results are more conservative in performance terms but more explicit in runtime assurance (Eichelbeck et al., 2024). Optimal control consistently outperformed RL in the reported low-voltage network experiments, yet the RL performance gap shrank under noisy forecasts. The safety shield effectively prevented constraint violations by projection correction, and training curves showed that in decentralized MAPPO the number of action corrections per agent decreases over training and the constraint violation penalty shrinks. In the centralized PPO setting, the penalty reduced, but action corrections remained frequent due to stricter DC power flow constraints; in deployment, centralized RL actions were corrected in every time step for the illustrated day.

Open problems remain substantial. Standard RL agents trained without explicit constraints are vulnerable to small, black-box observation perturbations; experiments still focus mainly on observation attacks rather than action-channel integrity or coordinated multi-vector attacks; and there is a lack of certified guarantees at realistic grid scales (Chen et al., 2021). CommonPower’s current release does not yet integrate robust MPC or chance-constrained formulations, and its guarantees are only with respect to the symbolic model, so model conformance and abstraction quality affect real-world guarantees (Eichelbeck et al., 2024). The voltage-stability framework still shows occasional violations on rare outlier operating conditions, and its performance is sensitive to hyperparameters such as $|S_\ell| \leq S_\ell^{\max}$34, active-learning batch sizes, SAC temperature, and step-size scaling through $|S_\ell| \leq S_\ell^{\max}$35 (Bi et al., 2024).

Future directions are correspondingly diverse. The literature identifies scalable, certifiable guarantees through reachability or barrier-certificate methods coupled with constrained RL, coordinated multi-agent safety over DER fleets with networked constraints and protection coordination, bounded-latency shielding and MPC for large networks, secure MLOps with verifiable data provenance and continuous validation on digital twins, and joint cyber-physical design with protection settings and EMS or AGC logic (Chen et al., 2021). Additional directions include robust optimal control, hierarchical control, contingency constraints, explicit decentralized negotiation, multi-timescale stability, and certified defenses such as randomized smoothing and interval bound propagation (Eichelbeck et al., 2024, Bi et al., 2024). A plausible implication is that Leading-Power Safety is evolving toward a layered architecture in which data hygiene, constrained learning, runtime shielding, model-based fallback, and institutional governance are all treated as first-class safety mechanisms rather than auxiliary safeguards.