Physics-Informed Deep Neural Network Method for Limited Observability State Estimation

Abstract: The precise knowledge regarding the state of the power grid is important in order to ensure optimal and reliable grid operation. Specifically, knowing the state of the distribution grid becomes increasingly important as more renewable energy sources are connected directly into the distribution network, increasing the fluctuations of the injected power. In this paper, we consider the case when the distribution grid becomes partially observable, and the state estimation problem is under-determined. We present a new methodology that leverages a deep neural network (DNN) to estimate the grid state. The standard DNN training method is modified to explicitly incorporate the physical information of the grid topology and line/shunt admittance. We show that our method leads to a superior accuracy of the estimation when compared to the case when no physical information is provided. Finally, we compare the performance of our method to the standard state estimation approach, which is based on the weighted least squares with pseudo-measurements, and show that our method performs significantly better with respect to the estimation accuracy.

• The paper presents a physics-informed DNN that integrates power-flow constraints into its loss function to improve state estimation under sparse measurements.
• It leverages historical full observability along with a partially observed snapshot on the IEEE 37-node feeder, outperforming classical WLS methods.
• The methodology provides a computationally efficient real-time DSSE solution that mitigates sensor failures and guides future improvements in PMU integration.

Physics-Informed Deep Neural Networks for State Estimation under Limited Observability

Introduction and Problem Context

The state estimation (SE) problem in distribution grids remains essential for ensuring reliable operation, especially with rising penetration of inverter-based and highly variable renewable energy sources. Existing measurement infrastructure in distribution systems is typically sparse, resulting in limited observability where standard weighted least squares (WLS) approaches become under-determined and perform suboptimally. This paper introduces a physics-informed deep neural network (DNN) architecture for the distribution system state estimation (DSSE) problem under conditions of severe measurement incompleteness. The method leverages grid physical knowledge (line and shunt admittance structure) directly in the learning objective, thereby regularizing the network's search space and enhancing convergence and accuracy, particularly in challenging unobservable scenarios.

Methodology Overview

The proposed approach is grounded in a hybrid ML-physics paradigm. The authors reformulate the DSSE problem for partial observability: leveraging $T-1$ fully observable historical snapshots and a partially observed present, the task is to estimate the full voltage phasor vector at the current time. The method introduces a DNN architecture composed of two primary modules: a feature extractor (incorporating LSTM and fully connected layers to process historical and partially observed data, respectively), and a regressor to output the complete state estimate.

A critical innovation lies in the loss function:

$$\mathcal{L}(\underline{s},\underline{v},\underline{\hat{v}},Y,\lambda) = \|\underline{v} - \underline{\hat{v}}\|^2 + \lambda \|\underline{s} - \operatorname{diag}(\underline{\hat{v}}) Y^* \underline{\hat{v}}^*\|^2$$

The first term is the standard MSE, while the second penalizes physical infeasibility relative to the AC power-flow equations (PFE), weighted by coefficient $\lambda$. This dual-objective enforces stricter compliance with power system physics during DNN optimization.

Grid Topology, Data Preparation, and Training Setup

The method is validated on the IEEE 37-node test feeder, with real-world active and reactive power profiles extracted from feeder data in Anatolia, CA. Power phasors (loads and generator injections) are assigned across the grid, and corresponding voltage phasors are generated via full AC power flow simulation.

Figure 2: The IEEE-37 Node topology, with explicit load (red) and generator (green) bus assignments used for constructing measurements and targets.

The data is pre-processed by low-pass filtering (moving average) and aggressive downsampling (from 1 Hz to 1 min), significantly reducing noise and temporal redundancy, resulting in 10,080 samples per bus for experiments.

A tailored dataset preparation pipeline selects sequences of length $T$, consisting of $T-1$ fully observable states and one partially observed time step. Observability degradation is structured, removing power measurements bus-wise in feeder order to simulate plausible network contingencies.

Figure 4: Schematic of the data preparation pipeline, detailing sequence extraction, removal of measurements to simulate sudden loss, and dataset partitioning.

The network is trained over permutations of observability levels (expressed as the fraction of available power phasors, with voltage phasors entirely suppressed in the partial state), history length $T \in \{5, 50\}$, and PFE regularization weight $\lambda \in \{0, 1, 2, 20\}$. A baseline WLS scheme is implemented using pseudo-measurements derived from the latest available values.

Experimental Results

The evaluation metric focuses on MSE of the estimated voltage magnitude and angle, de-standardized and presented in physical units. The results systematically demonstrate that:

• DNN-based estimators consistently outperform WLS and persistence baselines at all observability levels, with the gap widening as measurement availability decreases.
• Physics-informed regularization ($\lambda > 0$) yields further improvements, especially in voltage angle estimation, a dimension often neglected but increasingly important in low-inertia inverter-rich grids.

  Figure 1: Illustration of a partially observed state in the IEEE-37 feeder: buses without power measurements after simulated failure highlighted in red.

This advantage holds robust for both short ($T=5$) and longer ($T=50$) history lengths, indicating minimal dependence on longer temporal windows under the smoothed dynamics considered. The detailed ablation confirms:

• Increasing the regularization parameter $\lambda$ reduces angle estimation errors more substantially than magnitudes, as the physics constraints primarily rectify rotational degrees of freedom otherwise unconstrained in the data-driven mapping.
• Further increments in look-back window $T$ offer negligible benefit indicating that, given sufficient smoothing, the DNN can effectively infer state transitions from very short recent histories.

Theoretical and Practical Implications

By embedding power flow physical constraints as loss regularizers, the approach narrows the hypothesis space—improving generalization under data scarcity and reducing variance attributable to spurious deep learning minima. In the practical context of DSSE, this translates to:

• Improved estimation fidelity even under abrupt, severe sensor or communication failures, directly addressing reliability requirements in critical infrastructure.
• The method circumvents the need for costly wide-area PMU deployment, enabling immediate retrofitting atop legacy measurement deployments.
• The DNN solution is computationally efficient at inference due to the forward nature of neural evaluation, offering scalability for real-time distribution automation applications.

Theoretically, this work demonstrates the concrete value of physics-informed regularization in ill-posed, under-determined system identification. Unlike prior work requiring careful PMU placement or heuristic regularization, hard-coded physical knowledge in the DNN loss directly optimizes over the physically meaningful solution set.

Limitations and Future Directions

Key areas identified for further research include rigorous selection of the optimal $\lambda$ as a function of observability, integrating advanced PMU data imputation schemes, extending the DNN architecture to natively handle complex-valued phasors, and training a unified model robust to a continuum of observability patterns without per-case retraining.

Conclusion

The paper establishes that physics-informed DNN architectures provide a marked improvement over classical and pure data-driven approaches for DSSE under extreme measurement loss. The addition of explicit physical regularization in deep learning enables accurate recovery of both voltage magnitudes and angles in under-determined regimes. This methodological fusion of system physics and representation learning foreshadows key developments in robust, real-time estimation and control for future autonomous power grids.