Residual-loss Anomaly Analysis of Physics-Informed Neural Networks: An Inverse Method for Change-point Detection in Nonlinear Dynamical Systems with Regime Switching

Abstract: Nonlinear dynamical systems with regime transitions are typically described by ordinary differential equations with jumping parameters parameters. Traditional methods often treat change-point detection and parameter estimation as separate tasks, ignoring the inherent coupling between them. To address this, we propose residual-loss anomaly analysis of physics-informed neural networks, a unified framework that leverages dynamical consistency within the physics-informed learning paradigm. This approach jointly infers piecewise parameters and transition points under a single set of constraints. The method follows a two-stage strategy: First, local physical residuals are analyzed through overlapping subinterval decomposition. When a subinterval spans a true transition point, the residual exhibits a distinct structural elevation in noise-free conditions, which has a non-zero lower bound, enabling effective localization of potential transition intervals. Second, within our framework, change-point locations and piecewise parameters are integrated into a unified physical loss function for joint optimization, enabling simultaneous identification. Experiments on benchmark nonlinear dynamical systems, including Malthusian and logistic growth models, Van der Pol oscillator, Lotka-Volterra model and Lorenz system, demonstrate that the proposed method outperforms traditional decoupled approaches in both change-point localization and parameter estimation accuracy. This study provides an efficient, unified solution for structurally coupled inverse problems in nonlinear dynamical systems with regime switching.

• The paper introduces RAA-PINNs, a two-stage method that uses elevated physics residuals to localize regime switches before jointly refining change points and piecewise parameters.
• Across five nonlinear ODE benchmarks, the method contained all true changes in candidate intervals and achieved change-point squared errors from 1.69×10⁻⁶ to 8.65×10⁻⁴, with parameter errors as low as approximately 10⁻⁶.
• The approach outperformed AGM-PELT on Lotka–Volterra parameter recovery and achieved 79.5–103.9× parallel speedups, but its strongest guarantees remain limited to noise-free, identifiable, well-specified systems.

Problem setting and motivation

The paper addresses change-point detection in nonlinear dynamical systems (NDS-CPD): given partial observations of a trajectory governed by an ODE whose parameter vector switches discontinuously at unknown times, jointly recover both the switching locations and the piecewise-constant parameters. The authors argue that existing approaches—classical statistical methods (penalized likelihood, PELT, Bayesian reversible-jump MCMC) and data-driven methods (PINNs, SINDy)—share a common weakness: they decouple structural identification from parameter inference, either segmenting first and estimating second or vice versa. Because transition times and parameter values are structurally coupled—unknown jumps distort parameter estimation across regimes, while misestimated parameters obscure true transitions—decoupled pipelines undermine both consistency and stability.

The proposed framework, RAA-PINNs (residual-loss anomaly analysis of physics-informed neural networks), rests on a simple observation: when a single locally constant parameter is fit to an interval spanning a true regime switch, the resulting model misspecification manifests as an irreducible, structurally elevated physics residual. The residual anomaly thus serves as an endogenous detection signal that requires no prior segmentation of a pre-estimated parameter trajectory.

Mathematical formulation and well-posedness

The system is modeled as $$\mathrm{d}\boldsymbol{x}(t)=\boldsymbol{f}(t,\boldsymbol{x}(t);\boldsymbol{\theta}_{r(t)})\,\mathrm{d}t$$, where $r(t)$ is a right-continuous piecewise-constant discrete state taking finitely many values. A standard Picard–Lindelöf argument under Lipschitz and linear-growth conditions establishes existence and uniqueness of a piecewise continuously differentiable solution, constructed by concatenating solutions on each switching interval. This well-posedness result provides the foundation for treating the inverse problem as mathematically consistent.

Stage I: coarse localization via residual signatures

The temporal domain is partitioned into overlapping subintervals $I_k=[t_{k-1}-\delta,\,t_k+\delta]$, on each of which an independent local PINN with a constant parameter vector is trained. The normalized terminal physics-residual energy $S_k$ (a median over the last $M$ training iterations for robustness) is computed per subinterval, standardized via median absolute deviation, and subintervals exceeding a threshold are flagged as candidates.

The central theoretical contribution is a lower bound on the irreducible residual: under a parameter-affine vector field $\boldsymbol{f}=G\boldsymbol{\theta}+\boldsymbol{b}$ and a local identifiability condition ($\lambda_{\min}$ of the information matrix bounded below by $\alpha|J|$), any subinterval containing the change point satisfies

$$\underline{\mathcal{R}_k}\;\ge\;\alpha\,\frac{|I_k^-|\,|I_k^+|}{|I_k^-|+|I_k^+|}\,\|\boldsymbol{\theta}^- - \boldsymbol{\theta}^+\|_2^2,$$

whereas subintervals lying entirely within one regime admit zero residual in the noise-free case. The elevated residual on crossing intervals is therefore not an empirical artifact but a structural consequence of regime switching. Two caveats apply at this point: the bound holds only in the noise-free setting, and its translation to the learned discrete score $S_k$ relies on quadrature consistency, sufficient network capacity, and adequate training—the paper asserts these qualitatively rather than proving robustness of the score ordering under noise.

A complementary theorem shows that post-change residuals remain bounded below by the jump magnitude $r(t)$0 whenever this dominates trajectory approximation error $r(t)$1 and quadrature error, justifying residual-based localization after candidate transitions.

Stage II: joint refinement

Within the candidate interval $r(t)$2 from Stage I, the change point is reparameterized as $r(t)$3 to keep it interior, and the discontinuous step profile is replaced by a smooth sigmoid gate $r(t)$4. A local PINN is then trained jointly over $r(t)$5 under a unified loss balancing data misfit and gated physics residual. In the limit of large sharpness $r(t)$6, the surrogate approaches a hard switch. This converts the interval-level output of Stage I into a direct pointwise estimate of $r(t)$7 together with both regime-specific parameter vectors—an advantage over methods such as PINNs-EM-GMM that intrinsically produce only interval estimates.

Error analysis

Following conditional-stability-based generalization theory for PINNs, the paper derives a bound decomposing the generalization error into training residuals and quadrature errors:

$r(t)$8

This supports the reliability of the local inverse modules used in both stages, and defines a "well-trained" regime in which quadrature, not optimization, dominates the generalization gap. The analysis assumes a conditional stability estimate holds; it is not derived from scratch for the specific systems considered.

Numerical results

Experiments cover five benchmarks: Malthus growth, logistic growth, Van der Pol oscillator, Lotka–Volterra, and Lorenz. Networks are 4-layer MLPs (64 neurons, tanh), trained with Adam; Stage II uses a three-step schedule (pre-training, change-point optimization, joint optimization).

Localization accuracy: all candidate intervals from Stage I fully contain the true change points (e.g., $r(t)$9 for a change at $I_k=[t_{k-1}-\delta,\,t_k+\delta]$0), and Stage II refines them to squared errors as small as $I_k=[t_{k-1}-\delta,\,t_k+\delta]$1 (Van der Pol) and up to $I_k=[t_{k-1}-\delta,\,t_k+\delta]$2 (worst case, Malthus). Parameter squared errors range from roughly $I_k=[t_{k-1}-\delta,\,t_k+\delta]$3 to $I_k=[t_{k-1}-\delta,\,t_k+\delta]$4 across regimes, including simultaneous recovery of four coupled Lotka–Volterra coefficients across five regimes and three Lorenz parameters under chaotic dynamics (restricted to $I_k=[t_{k-1}-\delta,\,t_k+\delta]$5 to control trajectory divergence—a practical limitation inherent to chaotic systems).

Comparisons: against AGM-PELT (adaptive gradient matching plus PELT) on Lotka–Volterra, RAA-PINNs achieves squared parameter errors two to three orders of magnitude smaller (e.g., $I_k=[t_{k-1}-\delta,\,t_k+\delta]$6 vs. $I_k=[t_{k-1}-\delta,\,t_k+\delta]$7 for $I_k=[t_{k-1}-\delta,\,t_k+\delta]$8 in one regime) with sharper transitions and less within-regime dispersion. Against PINNs-EM-GMM, parameter accuracy is comparable (both around $I_k=[t_{k-1}-\delta,\,t_k+\delta]$9–$S_k$0), but RAA-PINNs yields direct point estimates of change points whereas PINNs-EM-GMM produces credible intervals only.

Parallelization: because Stage I subinterval problems are uncoupled, they map naturally onto parallel workers. With 128 concurrent tasks on 8 GPUs, total speedups of 79.5–103.9× were observed (efficiencies 0.62–0.81), reducing wall-clock time from roughly 15–20 hours serially to under 15 minutes, without altering the mathematical formulation or results.

Limitations and open questions

The paper concedes several restrictions explicitly. The residual lower-bound theory is established in the noise-free setting; behavior of the score ordering under measurement noise is argued heuristically through Assumption 2 and sufficient training but not proven. The identifiability assumption requires parameter-affine dynamics with a uniform lower bound on information-matrix eigenvalues, which excludes degenerate or weakly identifiable regimes and does not hold for arbitrary nonlinear parameterizations. The generalization analysis presupposes a conditional stability estimate rather than establishing it. Experiments use synthetic data with known structure; performance under model misspecification, uncertainty quantification, and real observational data remains unexamined. The method also presumes a known number of regimes implicitly through the gating construction, and extension to multiple simultaneous change points within a single candidate interval, stochastic dynamics, PDEs, and high-dimensional coupled systems is left open.

Conclusion

RAA-PINNs unifies change-point detection and parameter estimation for regime-switching ODEs by exploiting residual-loss anomalies as intrinsic, theoretically grounded signals of structural mismatch. Its two-stage design—parallelizable coarse screening followed by differentiable joint refinement—yields pointwise change-point estimates and accurate piecewise parameters, outperforming decoupled statistical baselines substantially and hybrid PINN-statistical methods in localization precision. The main theoretical guarantees are confined to idealized conditions, and extending the framework's robustness guarantees to noisy, misspecified, and higher-dimensional settings constitutes the principal open problem raised by this work.