A Shallow Recurrent Decoder for Dynamic State Estimation with a Limited Number of PMUs in Power Systems

Abstract: Dynamic State Estimation (DSE) will play a fundamental role in future power system operation by providing real-time estimates of the system state and enabling enhanced situational awareness. Existing DSE approaches are primarily based on Kalman filter variants or Machine Learning (ML) techniques. However, Kalman-based methods often suffer from high computational complexity, sensitivity to model inaccuracies, and performance degradation under strongly nonlinear operating conditions. Moreover, their effectiveness critically depends on the number and placement of measurements, since suboptimal PMU locations can reduce observability and even render state estimation infeasible. Machine learning approaches alleviate some of these limitations but typically require large amounts of training data and may struggle to generalize. To address these challenges, this paper proposes a SHallow REcurrent Decoder (SHRED) architecture for full-state reconstruction of power systems from sparse measurements. Unlike conventional model-based estimators, the proposed approach does not rely on an accurate physical model and is largely insensitive to PMU placement, making it particularly attractive for practical deployment in existing Wide Area Measurement Systems (WAMS). The method is validated on the IEEE 39-bus system under strongly nonlinear conditions, including short-circuit disturbances. The results demonstrate that SHRED can accurately reconstruct the complete system state using only a limited number of PMU measurements, consistently outperforming a state-of-the-art shallow decoder benchmark in sparse-measurement scenarios. Furthermore, the proposed framework exhibits strong robustness to measurement noise and maintains high reconstruction accuracy even under severe disturbances, highlighting its potential as a scalable and reliable alternative to conventional DSE techniques.

• The paper applies an LSTM-based SHRED architecture to reconstruct all generator and bus states in the IEEE 39-bus system from as few as one or two PMUs, addressing observability limits in conventional DSE.
• SHRED achieves mean relative errors of 1.35% with 39 PMUs, 1.75% with five, 3.40% with two, and 3.79% with one, while maintaining inference times as low as 1.3 ms and remaining robust to substantial measurement noise.
• The results show that temporal encoding enables accurate estimation across varied PMU placements, but simulation-dependent training, testing on only one system, and uncertain transfer to real grids remain key limitations for deployment.

The SHallow REcurrent Decoder (SHRED) architecture, originally developed for sparse sensing and state reconstruction in scientific dynamical systems, is applied here for the first time to Dynamic State Estimation (DSE) in power systems (2607.00116). The paper addresses a persistent practical gap: model-based DSE methods such as Kalman filter variants require full system observability, which is rarely achievable because Phasor Measurement Units (PMUs) are deployed by Transmission System Operators (TSOs) for objectives unrelated to observability—oscillation monitoring, voltage assessment—and are seldom installed at generator terminals. The proposed approach reconstructs the complete dynamic state of the IEEE 39-bus system, including all synchronous generator states and bus quantities, from as few as one or two PMU channels, while remaining largely insensitive to PMU placement and robust to measurement noise.

Motivation and positioning

DSE relies on the discretized measurement equation derived from the system's differential-algebraic equations, tracking generator rotor speeds, angles, and damper/excitation fluxes to support Dynamic Security Assessment, out-of-step protection, and voltage instability monitoring. Classical estimators—the Extended Kalman Filter (EKF) and Unscented Kalman Filter (UKF)—suffer from linearization errors or computational burden under strong nonlinearities, and critically depend on optimal PMU placement; without observability, estimation becomes infeasible. Existing machine learning alternatives require large datasets and extensive tuning.

A notable methodological choice is that the paper deliberately does not benchmark against KF/EKF/UKF. The authors argue that these methods perform well only under ideal, fully observable measurement configurations, whereas the objective is precisely the regime where such assumptions fail. This framing is defensible but means the claimed advantage over model-based estimators is asserted rather than demonstrated numerically.

Architecture

SHRED couples an LSTM encoder with a Shallow Decoder Network (SDN). The LSTM maps a window of $l$ time lags of PMU measurements ($V$, $f$, $\theta$ at each instrumented bus) into a latent representation—a construction motivated by Takens' embedding theorem—and the SDN decodes this latent vector into the full state space. Training is performed not on the high-dimensional state directly but on an SVD-compressed representation retaining 99% of cumulative spectral energy, which reduces training cost sufficiently that models train in minutes on commodity hardware.

Training data consist of 680 DIgSILENT PowerFactory RMS simulations of the IEEE 39-bus system spanning 15 network configurations and fault clearing times from 20 ms to 200 ms, split 80/10/10 into training/validation/test sets. Outputs comprise six generator states per machine ($\omega$, $\delta$, $\psi_{1d}$, $\psi_{1q}$, $\psi_{2q}$, $\psi_{\mathrm{exc}}$) plus $V$0, $V$1, and $V$2 at all buses.

Sensitivity to the number of PMUs

Eight case studies progressively reduce PMU coverage from all 39 buses down to a single bus at the highest short-circuit power $V$3. Evaluated on held-out fault scenarios via the relative $V$4 error per variable class, the results are strong:

Case study	PMUs	Mean $V$5 [%]
A1	39	1.352
A3	20	1.656
A5	5	1.749
A6	3	2.113
A7	2	3.401
A8	1	3.790

Across the entire test set, reconstruction error stays below 1% for frequency, rotor speed, and all flux variables, below 2% for voltage magnitudes, and below 4% for rotor angle $V$6 provided at least three PMUs are available. Voltage angle $V$7 is consistently the worst-reconstructed variable class, attributed to its strong dependence on network configuration. Accuracy degrades noticeably only in the two-PMU and one-PMU cases, where specific flux and speed trajectories show larger deviations.

Against a baseline SDN operating directly on instantaneous measurements, SHRED's advantage grows monotonically as measurements become sparser; with abundant sensors both perform comparably, but the LSTM's temporal encoding preserves accuracy in low-measurement regimes. This is the paper's clearest empirical claim: temporal context, not architectural depth, drives performance under sparse instrumentation.

Insensitivity to PMU placement

Nine placement clusters—three with 10 PMUs and six with 5 PMUs, restricted to HV buses since MV generator buses are typically inaccessible to TSOs—are evaluated. Mean relative errors range narrowly from 1.42% to 2.20%, confirming near-agnosticism to location. This is the property with the most direct operational significance: configurations that would render Kalman-based DSE infeasible remain usable with SHRED. Additionally, ensembling reconstructions across clusters yields better accuracy than any individual cluster, motivating ensemble deployment.

Robustness to noise and computational cost

With Gaussian noise at $V$8 relative to each signal's standard deviation—a substantial contamination level—applied to three 10-PMU clusters, mean errors (1.54%, 1.54%, 1.84%) remain essentially unchanged from the noise-free case, with only mild degradation in bus-variable reconstruction. The ensemble mean again outperforms individual models.

Inference averages 1.3 ms (range 0.6–3.1 ms) on an NVIDIA T600 GPU, comfortably faster than the 20 ms WAMS sampling period used in Europe; offline training averages roughly 13.5 minutes. Real-time feasibility is therefore well supported.

Limitations and open questions

The principal limitation, acknowledged plainly by the authors, is that SHRED is supervised and requires full-state trajectories for training—quantities unmeasurable in real grids. The proposed remedy is Data Assimilation SHRED (DA-SHRED), trained on high-fidelity simulation models that TSOs already maintain for DSA, then deployed against real PMU streams; however, this transfer from simulated to real dynamics remains unvalidated in the power system context and constitutes the central open question. Two further directions remain open: integration with SINDy for interpretable latent-space governing equations, and validation on real-world systems during severe disturbances. It should also be noted that all results derive from a single test system (IEEE 39-bus), so scalability to interconnection-scale networks is untested.

Conclusion

This work demonstrates that a shallow recurrent decoder can perform full-state DSE on the IEEE 39-bus system with mean relative errors below 2% using five or more PMUs, below 4% with a single PMU, across random placements and under significant measurement noise, with millisecond-scale inference. Its practical contribution lies in relaxing the observability requirements that constrain conventional estimators, aligning DSE capability with realistically deployed WAMS infrastructure. The decisive remaining question is whether simulation-trained DA-SHRED retains this accuracy when confronted with real measurement streams and unmodeled dynamics.