Grid2Op Framework for Power Grid Simulation

• Grid2Op is an open-source Python framework that simulates power-grid operations with high fidelity using modular design and a robust Markov Decision Process formulation.
• It integrates industry-grade AC solvers with a Gym-compatible interface to enable rapid prototyping, benchmarking, and testing of reinforcement learning and control strategies.
• The framework supports extensive scenario generation, adversarial modeling, and reproducible benchmarking, advancing research in resilient and adaptive grid operation.

Grid2Op is an open-source Python framework for simulating and operating electrical power networks in a sequential, realistic, and physically accurate fashion. Designed foremost for research into reinforcement learning (RL), robust control, and optimization of power-grid operation, Grid2Op models the real-time decision-making of transmission system operators managing uncertainties from variable generation, fluctuating demand, and network contingencies (Marot et al., 2021). It bridges the gap between academic power grid research and operational constraints encountered in large-scale, nonlinear grids by exposing a modular, extensible, and fully-featured environment tailored for rapid prototyping, benchmarking, and deployment of AI-based control policies.

1. Architecture and Design Principles

Grid2Op is structured around modular building blocks:

• Simulator interface: Abstracts power-flow computations, delegating to industry-grade AC network solvers (e.g., pandapower, custom C++ engines), thus eschewing the simplifications common in traditional OPF research. This enables accurate modeling of voltage, phase, and current considering AC nonlinearity.
• State representation: Encapsulates all observable and latent network quantities, including topology, line flows, generator/consumer injections, breaker statuses, timer/cooldown information, and temporal context (calendar features).
• Action interface: Unifies topological interventions (line switching within substations) and continuous actions (generation redispatch), exposing a combined discrete–continuous space that reflects the operational levers available to human operators.
• MDP wrapper: Presents an OpenAI Gym–compatible interface, including methods for state reset, step transitions, and an offline action simulation API that enables “what-if” scenario analysis for safety or human-in-the-loop workflows.

The principal design objectives include maximal modularity (plug-in backends, scenario generators, reward wrappers), the ability to accommodate very large and complex action/state spaces (e.g., $>70,000$ topologies and 40+ generators), and representativeness with respect to operational power-system challenges (Marot et al., 2021).

2. Markov Decision Process Formulation

Grid2Op formalizes grid operation as a Markov Decision Process (MDP):

• States $s_t$: Each state aggregates current topology ($\tau_t$), power injections $\{P_i(t),Q_i(t)\}$ for all buses, line power flows $\{F_\ell(t)\}$, disconnection/maintenance timers, and auxiliary calendar features. The power flow at each time obeys the nonlinear AC equations:

$$P_i = \sum_{j=1}^n |V_i||V_j| \left(G_{ij} \cos(\theta_i-\theta_j) + B_{ij} \sin(\theta_i-\theta_j)\right)$$

$$Q_i = \sum_{j=1}^n |V_i||V_j| \left(G_{ij} \sin(\theta_i-\theta_j) - B_{ij} \cos(\theta_i-\theta_j)\right)$$

• Actions $a_t$: Composed of discrete substation switching operations and continuous generator redispatch within ramp/capacity constraints. The line switching combinatorics can reach $>70,000$ due to simultaneous breaker choices.
• Transition function: Applies $a_t$ under feasibility, rate, and downtime constraints; samples stochastic exogenous injections (load/renewables); processes contingencies like adversarial tripping and scheduled maintenance; then runs the AC power-flow to determine the new state and trips lines overloaded beyond thermal limits, which can trigger cascading outages.
• Reward $r(s_t, a_t)$: Penalizes load-shedding severely (ending the episode), and otherwise accounts for line overloads, redispatch, and losses. During competitions, reward design typically maximizes survival time (steps before blackout), with a normalized leaderboard mapping (e.g., $[-100, 0, 80, 100]$ for immediate blackout, baseline, survival, and cost savings) (Marot et al., 2021).

Grid2Op supports both episodic and continuous control paradigms; episodic rollouts (e.g., $T=2016$ steps per episode) simulate week-long operational windows with five-minute dispatch granularity.

3. Scenario Generation and the GridAlive Ecosystem

The GridAlive companion toolkit supports high-level scenario and environment construction:

• Network topology definition/import: Users can define or import bus–branch models from standards (IEEE, Matpower, PSS/E).
• Chronics generation: Synthetic or real, temporally and spatially correlated time series for loads and renewable generation.
• Scenario pipelines: Users can encode maintenance schedules, stochastic/adversarial attacks, weather events, hardware upgrades (storage, DERs) via a plugin interface.
• Visualization/export: Scenarios and chronics can be output in JSON or pickled format for downstream use.

This enables systematic benchmarking across diverse operational contexts, from high-renewable to stressed or adversarial settings (Marot et al., 2021).

4. Supported Perturbations and Adversarial Modeling

Grid2Op natively encodes multiple types of operational uncertainties and perturbations, crucial for RL research:

• Deterministic outages: Time-scheduled line unavailability to simulate maintenance.
• Stochastic and adversarial line tripping: Random or targeted line removals reflecting physical faults or cyber attacks, e.g., adversary selectively disconnecting highly loaded lines at random intervals.
• Renewable and load uncertainty: Chronics spanning diverse renewable penetrations (10–30%) with train/test splits to assess generalization, and statistically modeled load variations.
• Custom scenario objects: All perturbations are specified via scenario objects passed at environment instantiation, guaranteeing reproducibility and extensibility for research (Marot et al., 2021, Dwivedi et al., 2024, Peter et al., 2024).

5. Benchmarking Suite: Metrics, Baselines, and Evaluation Protocols

Grid2Op provides a comprehensive suite for reproducible benchmarking:

Component	Details	Reference Agents
Scenario corpus	>100,000 years of simulated chronics for training; 24 test scenarios per eval run	DoNothing, RuleBased, AdaptiveExpert, baseline RL (DQN/PPO)
Metric suite	Survival time (steps to blackout/episode end); normalized reward
Leaderboard	Score mapping: blackout $-100$, baseline $0$, survival $80$, cost optimal $100$
Competition flows	Three phases: warm-up, feedback, final test on held-out scenarios; time/compute constraints

Baseline policies encode both rule-based heuristics and state-of-the-art RL agents; the Grid2Op baselines repository includes implementations for DQN, PPO, and imitation learning for direct comparison (Marot et al., 2021).

6. Implementation API and Extensibility

The core API exposes:

• Environment instantiation: grid2op.Env and grid2op.make (customizable via YAML/JSON scenario files, backend choice, random seed control).
• Simulator backend: plug-in support (pandapower, C++ solver) via subclassing.
• Step and reset interface: Gym-style env.reset(), env.step(action), augmented by env.simulate(action) for off-policy checking.
• Observation/action objects: Support both vectorized and graph-based representations; helper factories enable efficient action-space coverage.
• Full Gym/RLlib compatibility: Designed to integrate with common RL experiment management and distributed training infrastructure.
• Custom wrappers: Direct user support for new event generators (BasePerturbation), new RL paradigms (multi-agent, human-in-the-loop via Gym wrappers), scenario batch evaluation, and reward customization (Marot et al., 2021).

Installation is available via PyPI with backend-specific extras.

7. Research Applications and Extensions

Grid2Op has catalyzed a broad research ecosystem in robust and adaptive grid operation:

• Physics-Guided RL: Integrates power-flow sensitivity (LODF) into RL exploration, dramatically improving sample efficiency and resilience in blackout mitigation tasks (Dwivedi et al., 2024).
• Dual-Policy and Adversarial RL: Facilitates multi-agent scenarios and N–$k$ contingency screening, jointly optimizing defensive control against learned/adversarial attacks (Peter et al., 2024).
• Graph-based Distributed RL: Enables decomposition of observation and action spaces for distributed control via GNNs, improving scalability and generalization (Fabrizio et al., 2 Sep 2025).
• Safe RL and Multi-Objective Extensions: Ongoing work includes formal safety guarantees (e.g., provable no-load-shedding), economic cost integration, and continual learning under distributional shifts (Marot et al., 2021).

Winning solutions in recent benchmarks hybridize RL with expert rules, incorporate graph neural architectures, evolutionary policy search, and leverage the simulation API for online action validation (Marot et al., 2021).

8. Impact, Lessons Learned, and Future Directions

Grid2Op, through its modularity and realism, has enabled large-scale RL benchmarks (notably the NeurIPS L2RPN series), robust cross-scenario evaluation, and reproducible research in adaptive power system control. The software’s architecture supports extensions to meshed distribution grids, HVDC links, multi-agent markets, and cyber-resilient grid operations. Proposed research avenues include:

• Adding differentiable or surrogate physics layers for RL acceleration
• Embedding richer security and economics into the RL reward structure
• Human–machine teaming in grid operations, integrating interpretability and trust (Marot et al., 2021)

In summary, Grid2Op and GridAlive provide a physically accurate, extensible, and open-source foundation for sequential grid-operation research. They have become the de facto standard for RL benchmarking in power systems, accelerating innovations in robust, scalable, and intelligent control (Marot et al., 2021, Dwivedi et al., 2024, Peter et al., 2024, Fabrizio et al., 2 Sep 2025).