SHRED: Shallow Recurrent Decoder

• The paper demonstrates that SHRED uses a recurrent LSTM encoder coupled with a shallow decoder to robustly reconstruct high-dimensional states from sparse sensor data.
• It decouples temporal encoding from spatial compression by employing a two-level latent structure with LSTM and SVD, mitigating sensitivity to PMU sparsity and placement.
• Empirical results reveal low mean relative errors (below 4%) even with few PMUs, highlighting its potential for real-time dynamic state estimation in power systems.

Searching arXiv for SHRED and closely related papers to ground the article in current literature. SHallow REcurrent Decoder (SHRED) denotes a family of data-driven architectures for reconstructing high-dimensional dynamical states from sparse measurement histories, with subsequent domain-specific variants for reduced-order modeling, data assimilation, scientific discovery, and uncertainty quantification. In the power-systems formulation introduced for Dynamic State Estimation (DSE), SHRED is a sequence-to-state mapper that takes a short temporal window of PMU measurements from a limited set of buses, encodes that history with an LSTM, predicts a compressed representation of the full power-system state, and lifts it to the original state space through an SVD basis (Pomarico et al., 30 Jun 2026). More broadly, SHRED has been developed as a decoder-oriented sensing framework in which temporal structure, rather than dense spatial observation, provides the principal source of information for full-state reconstruction under sparse sensing (Williams et al., 2023).

1. Origins and conceptual definition

SHRED was introduced as a sensing architecture for inferring and forecasting high-dimensional spatiotemporal fields from very few, noisy, and possibly poorly placed sensors (Williams et al., 2023). In that formulation, the central idea is to replace static sensor-to-state regression by a mapping from a time-delay window of sensor measurements to a latent temporal representation, followed by a shallow decoder into the full spatial state. The underlying rationale is explicitly linked to Takens-style delay embedding: temporal trajectories at a few sensors can encode the attractor structure of the underlying dynamics (Williams et al., 2023).

Subsequent work generalized SHRED beyond its original sparse-sensing setting. A data-assimilation extension, DA-SHRED, uses a simulation-trained SHRED latent space and updates it with real sensor data to bridge the simulation-to-real gap, while also supporting discrepancy modeling through SINDy in latent space (Bao et al., 1 Dec 2025). Other variants include SINDy-SHRED and Koopman-SHRED for sparse discovery of nonlinear or linear latent dynamics (Gao et al., 23 Jan 2025), T-SHRED for transformer-based temporal encoding and symbolic-regression regularization (Yermakov et al., 18 Jun 2025), UQ-SHRED for distributional reconstruction via engression (Gao et al., 1 Apr 2026), and CS-SHRED for reconstruction under incomplete and corrupted measurements via an embedded compressed-sensing stage (Silva et al., 30 Jul 2025). PySHRED systematizes these developments in a software package for sparse sensing, model reduction, and scientific discovery (Ye et al., 28 Jul 2025).

Within power systems, SHRED was specialized to DSE in order to reconstruct the full dynamic state of a transmission network from sparse PMU measurements, explicitly addressing limitations of Kalman-filter-based methods under strong nonlinearities, model inaccuracies, and poor observability induced by sparse or suboptimal PMU placement (Pomarico et al., 30 Jun 2026). In that setting, SHRED is presented as a model-free-at-inference alternative that remains effective under sparse measurement configurations and severe disturbances.

2. Architecture and mathematical structure

In the power-systems DSE formulation, SHRED operates on a measurement window of length $l+1$,

$$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$

where each PMU provides $V$, $f$, and $\theta$ at its bus (Pomarico et al., 30 Jun 2026). The architecture is decomposed into a recurrent encoder $\mathcal{G}_1$ and a shallow feedforward decoder $\mathcal{F}_1$, yielding

$\mathbf{\hat{x}(t_k) = \mathcal{F}_1\big(\mathbf{z}(t_k)\big) = \mathcal{F}_1\left( \mathcal{G}_1\big(\mathbf{s}_i(t_k), \dots, \mathbf{s}_i(t_{k-l})\big) \right),$

with $\mathcal{G}_1$ implemented as an LSTM-based recurrent encoder and $\mathcal{F}_1$ as a shallow decoder network (Pomarico et al., 30 Jun 2026).

The recurrent encoder maps the sensor history to a latent vector

$$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$0

and in the power-system implementation uses an LSTM with two hidden layers, each with 64 neurons (Pomarico et al., 30 Jun 2026). The feature dimension at the input is $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$1, reflecting the three PMU observables per measured bus. The decoder then maps $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$2 to a compressed state using two hidden layers with 350 and 400 neurons (Pomarico et al., 30 Jun 2026). The designation “shallow” refers to the limited depth of this decoder, in contrast to very deep architectures.

A defining property of SHRED is its decoder-only orientation in the spatial sense. The broader literature emphasizes that SHRED is not a standard autoencoder: it does not learn an encoder–decoder pair on full fields, but rather learns a forward map from sparse sensor histories to a latent state and then from that latent state to the full field (Bao et al., 1 Dec 2025). This separation is repeatedly framed as a numerically favorable alternative to learning an ill-posed inverse mapping from full states to latent coordinates (Bao et al., 1 Dec 2025, Ye et al., 28 Jul 2025).

The power-systems paper also introduces a second latent layer through SVD compression. If the full-state snapshot matrix is

$$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$3

with SVD

$$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$4

then the first $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$5 left singular vectors $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$6 define a reduced subspace chosen to capture $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$7 of spectral energy (Pomarico et al., 30 Jun 2026). The compressed coordinates are

$$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$8

and SHRED is trained to predict $$\big( \mathbf{s}_i(t_{k-l}), \dotsc, \mathbf{s}_i(t_k) \big),$$9, followed by lifting

$V$0

This yields a two-level latent structure: the LSTM latent $V$1 encodes temporal measurement structure, whereas the SVD latent $V$2 compresses the full state (Pomarico et al., 30 Jun 2026). A plausible implication is that SHRED’s regularization derives partly from this explicit separation between temporal encoding and spatial compression.

3. Dynamic state estimation in power systems

The DSE application treats the underlying power-system dynamics as a DAE system,

$V$3

with measurements

$V$4

at discrete time $V$5 (Pomarico et al., 30 Jun 2026). In the IEEE 39-bus study, the dynamic state vector comprises generator dynamic states from a 6th-order synchronous-machine model—$V$6, $V$7, $V$8, $V$9, $f$0, and $f$1—together with bus-level $f$2, $f$3, and $f$4 for all buses (Pomarico et al., 30 Jun 2026).

This choice of target state is significant because it makes SHRED a full-state estimator rather than a bus-only regressor. The reconstructed state includes both algebraically visible variables and internal generator states that are generally not directly measured in practice (Pomarico et al., 30 Jun 2026). The paper explicitly motivates this by reference to dynamic security assessment, transient stability, wide-area protection, model validation, and WAMS situational awareness.

The training data are generated entirely from dynamic simulations of the IEEE 39-bus New England system in DIgSILENT PowerFactory, with 15 different network configurations, line short-circuit faults with clearing times from 20 ms to 200 ms, and 680 dynamic simulations in total (Pomarico et al., 30 Jun 2026). The split is 80% training, 10% validation, and 10% test. The full-state trajectories and PMU measurements are extracted from these simulations, and the PMU input consists only of $f$5, $f$6, and $f$7 at instrumented buses (Pomarico et al., 30 Jun 2026).

Training is performed in reduced space with an MSE loss,

$f$8

The paper identifies the reduced-space formulation itself as the main mechanism for lowering dimensionality and training cost (Pomarico et al., 30 Jun 2026). No explicit physics constraints are imposed in the loss.

4. Empirical behavior under sparse sensing

A central claim in the power-systems study is that SHRED is largely insensitive to PMU sparsity and placement (Pomarico et al., 30 Jun 2026). This is assessed in two ways: varying the number of PMUs and varying their locations.

In case studies A1–A8, the number of PMUs ranges from all 39 buses down to a single PMU (Pomarico et al., 30 Jun 2026). The mean relative $f$9 error is defined per variable class by

$\theta$0

Average errors remain low even under extreme sparsity. The reported mean $\theta$1 values are 1.352% for A1, 1.780% for A2, 1.656% for A3, 1.640% for A4, 1.749% for A5, 2.113% for A6, 3.401% for A7, and 3.790% for A8 (Pomarico et al., 30 Jun 2026). The paper states that reconstruction remains excellent down to 3 PMUs and that even with 1 PMU the average error remains below 4% over all variables and test cases.

The variable-wise behavior is uneven. For $\theta$2, $\theta$3, $\theta$4, $\theta$5, $\theta$6, and $\theta$7, the reported $\theta$8 remains below 1% across all cases; for bus voltage magnitude $\theta$9, below 2%; and for $\mathcal{G}_1$0, below 4% when at least 3 PMUs are available (Pomarico et al., 30 Jun 2026). Only $\mathcal{G}_1$1 exhibits visibly higher error, attributed in the paper to its dependence on network topology and global phase.

Placement sensitivity is examined through nine PMU-location clusters B1–B9 consisting of random subsets of high-voltage buses, with 10-PMU and 5-PMU cases and no PMUs placed at generator medium-voltage buses (Pomarico et al., 30 Jun 2026). The average $\mathcal{G}_1$2 values reported are 1.546%, 1.421%, 1.434%, 1.747%, 1.622%, 2.106%, 2.204%, 2.133%, and 1.972% for B1 through B9, respectively. The paper interprets the mild variation across clusters as evidence that SHRED is “largely agnostic to PMU placement” (Pomarico et al., 30 Jun 2026).

This placement robustness is consistent with the original sparse-sensing SHRED literature, where random sensor placements were found to yield performance similar to QR-based placements in turbulence, SST, and ozone reconstruction tasks (Williams et al., 2023). In that earlier work, the main performance gain over static baselines was traced to temporal encoding rather than optimized sensor location. This suggests a shared mechanism across domains: delay-embedded sensor histories can compensate, at least partially, for poor instantaneous observability.

5. Relation to baselines and adjacent methodologies

The power-systems SHRED paper positions itself against two broad classes of alternatives: Kalman-based DSE and machine-learning estimators (Pomarico et al., 30 Jun 2026). EKF and UKF are described as vulnerable to model inaccuracies, Gaussian noise assumptions, covariance tuning, computational burden, and especially observability degradation under sparse PMU placement (Pomarico et al., 30 Jun 2026). For this reason, the detailed experimental baseline is not EKF or UKF but a state-of-the-art shallow decoder network (SDN) lacking the recurrent encoder.

The baseline SDN shares the same decoder architecture as SHRED but maps measurements directly to the SVD coefficients without temporal modeling (Pomarico et al., 30 Jun 2026). The comparison shows that with many PMUs SDN and SHRED behave similarly, whereas under decreasing PMU counts SDN errors increase substantially and SHRED retains much lower error (Pomarico et al., 30 Jun 2026). The stated interpretation is that the recurrent encoder is crucial under partial observation because it exploits delay-embedded measurement history.

That interpretation aligns with the broader SHRED literature. The original sensing paper contrasted SHRED with linear POD/gappy-POD/QR-POD and with static shallow decoders, arguing that SHRED constitutes a nonlinear, data-driven generalization of separation of variables in which latent temporal structure and shallow spatial decoding are decoupled (Williams et al., 2023). Later work on SINDy-SHRED framed the decoder as the minimal scaffold on which symbolic latent dynamics can be imposed, thereby combining sparse sensing with interpretable dynamical models (Gao et al., 23 Jan 2025). T-SHRED, in turn, replaces the recurrent encoder by a transformer and adds SINDy-attention, although its own experiments report that GRU-based SHRED still achieves the best raw next-step test MSE across the studied datasets (Yermakov et al., 18 Jun 2025).

The relation to data assimilation is more subtle. DA-SHRED uses the SHRED latent basis learned from simulations and adapts it to real sensor data by weight updates or latent transfer operators, rather than using a classical probabilistic filter (Bao et al., 1 Dec 2025). This suggests a conceptual continuity with the power-systems application: the 2026 DSE formulation is not yet an assimilation framework in the DA-SHRED sense, but its reliance on simulation-derived full-state labels and reduced latent spaces makes such an extension natural.

6. Robustness, computational properties, and extensions

The power-systems study evaluates robustness to severe disturbances and measurement noise. Disturbance scenarios include line short-circuit faults, 20–200 ms clearing times, and 15 distinct network configurations (Pomarico et al., 30 Jun 2026). Testing also includes faults not seen in training. For measurement noise, PMU inputs are perturbed as

$\mathcal{G}_1$3

with $\mathcal{G}_1$4 in the reported experiments (Pomarico et al., 30 Jun 2026). For the 10-PMU clusters B1–B3, mean $\mathcal{G}_1$5 under noise is 1.539%, 1.541%, and 1.844%, compared with 1.546%, 1.421%, and 1.434% in the noise-free setting (Pomarico et al., 30 Jun 2026). The reported degradation is minor.

Computationally, the framework is lightweight at inference. In the IEEE 39-bus experiments, average training time is approximately 13.5 minutes, with a range of 11.9–17.5 minutes on an NVIDIA T600 GPU, and average inference time is about 1.3 ms per sequence, with a range of 0.6–3.1 ms (Pomarico et al., 30 Jun 2026). Since the PMU sampling period in Europe is about 20 ms, the paper concludes that SHRED is real-time capable.

The broader SHRED ecosystem has expanded these computational ideas in several directions. Parametric SHRED formulations for nuclear and fusion applications combine SVD-based reduction with recurrent sensor encoders to generalize across parameterized trajectories, often using only three sensors and reporting rapid training on workstation- or laptop-scale hardware (Riva et al., 11 Mar 2025, Verso et al., 2 Apr 2026, Verso et al., 11 Mar 2026). UQ-SHRED turns SHRED into a conditional distributional model by injecting noise into the sensor inputs and minimizing an energy-score loss, thereby producing predictive distributions rather than point estimates (Gao et al., 1 Apr 2026). CS-SHRED incorporates a compressed-sensing recovery stage and an SNR-guided adaptive loss to handle incomplete and corrupted measurements (Silva et al., 30 Jul 2025). PySHRED consolidates many of these variants and emphasizes compressive training, mobile sensors, parametric data managers, and latent forecasters (Ye et al., 28 Jul 2025).

For power systems, the main limitations stated for the 2026 DSE variant are that supervised training requires full-state trajectories, which in practice must come from simulations; training therefore depends on the fidelity of the underlying dynamic model even though inference does not (Pomarico et al., 30 Jun 2026). The framework is also trained on specific network configurations and operating ranges, and it imposes no explicit physics constraints such as Kirchhoff’s laws or stability conditions (Pomarico et al., 30 Jun 2026). The authors therefore propose future directions including DA-SHRED-style assimilation of real sensor data, integration with SINDy for interpretable latent dynamics, validation on real PMU data, and extension to larger grids and inverter-dominated distribution systems (Pomarico et al., 30 Jun 2026).

Taken together, these results position SHRED not as a single fixed network, but as a modular family of shallow recurrent decoders centered on sparse-sensor delay embeddings, low-dimensional latent representations, and shallow state decoders. In the specific context of power-system DSE, its distinguishing characteristic is the ability to reconstruct the full dynamic state—including generator internal variables and bus states—from sparse PMU histories with low error, limited sensitivity to PMU placement, robustness to noise and faults, and inference times well below PMU sampling intervals (Pomarico et al., 30 Jun 2026).