Model-free Anomaly Detection for Dynamical Systems with Gaussian Processes

Abstract: In this paper we address the problem of detecting differences or anomalies in a dynamical system, based on historical data of nominal operations. This problem encompasses quality control, where newly manufactured systems are tested against desired nominal operations, and the detection of changes in the dynamics due to degradation or repairs. We propose a model free approach based on Gaussian processes (GPs). The idea is to train offline a GP based on nominal data, which is then deployed online to detect whether measurements of the system state are compatible with nominal operations or if they deviate. Detecting this deviation is made more challenging by the presence of process and measurement noise, which might obfuscate deviations in the dynamics. The detection then is based on a threshold that ensures a specific false positive rate. We showcase the promising performance of the proposed method with two systems, and highlight several interesting future research questions.

• The paper introduces a GP-based approach that leverages residual analysis for model-free anomaly detection, achieving statistical control over false positives.
• The method employs a two-stage pipeline with GP system identification and online residual-based hypothesis testing to distinguish nominal from anomalous behavior.
• Empirical evaluations on a damped pendulum and a Van der Pol oscillator demonstrate up to 80% detection probability under low false positive thresholds.

Model-Free Anomaly Detection in Dynamical Systems via Gaussian Processes

Problem Statement and Motivation

This paper addresses the fundamental challenge of model-agnostic anomaly detection in dynamical systems when only data from nominal operation are available. Traditional discriminant analysis and quality control in such settings have relied on prescribed models, but in applications where system dynamics cannot be fully characterized due to complexity or nonstationarity, these approaches are inadequate. Conventional model-based identification techniques introduce parametric biases and frequently fail to capture epistemic uncertainty, compromising robust statistical guarantees.

The core contribution is the formulation of a flexible, data-driven method utilizing Gaussian Processes (GPs) for residual-based anomaly detection, explicitly quantifying uncertainty stemming from measurement noise, process noise, and function approximation. The approach provides statistical control over the false positive rate, setting it apart from prevailing neural and classical machine learning techniques.

Methodology

The procedure features a two-stage pipeline:

1. GP-Based System Identification:

A GP is trained using state transition data from multiple sampled trajectories under nominal conditions. The GP (with a squared-exponential kernel) yields a non-parametric estimate of the discrete transition function $F_p(x)$ along with credible intervals via posterior covariance.

2. Online Residual-Based Discrimination:

To determine whether a new trajectory $Q$ is consistent with prior observations, its transition residuals with respect to the GP model, $$\epsilon = [\epsilon_1, ..., \epsilon_{n_{steps}-1}]$$, are computed. The residual at each timestep incorporates four components: process and observation noise, model misspecification (dynamics error), GP approximation error, and observation-induced GP error (via Taylor linearization and Jacobian computations). This comprehensive decomposition enables a principled derivation of the residual covariance matrix, crucial for hypothesis testing.

The test statistic is the normalized squared Mahalanobis norm of the trajectory's residual, compared against the null distribution defined by the nominal GP model and noise covariances. The resulting $p$-value enables practitioners to calibrate the classifier's threshold for a user-specified false positive rate.

Theoretical Insights

A rigorous probabilistic framework underpins each stage of the algorithm.

• Residuals are shown to be multivariate Gaussian with mean given by the deviation between the true unknown dynamics $F_Q(x)$ and the nominal GP mean, and covariance a function of GP posterior variance, process, and observation noise.
• The stepwise analysis disentangles the influence of GP generalization error versus observational and process noise, providing actionable diagnostic leverage—uncertainty can be targeted via increased data collection (reducing GP variance), or by system design (reducing sensor/process noise).
• Parameter selection for the kernel hyperparameters is accomplished via maximum log-likelihood, optimizing GP fidelity with limited data (typically 10–30 short trajectories).

Empirical Evaluation

The method is validated on two nonlinear systems: a damped pendulum with a mass imbalance and a Van der Pol oscillator with parametric perturbation. Large ensembles of synthetic datasets (hundreds of thousands of samples) are used to produce robust estimations of the ROC curves under varying anomaly detection thresholds and time horizon lengths.

Strong numerical findings include:

• For the damped pendulum: At a false positive rate of 20%, the algorithm achieves a 60% true positive rate when analyzing short (10-step) trajectories. Performance degrades slightly when considering longer horizons, reflecting that discriminative deviations in dynamics exist mainly within certain temporal windows.
• For the Van der Pol oscillator: With a more pronounced dynamical deviation and a longer trajectory window (20 steps), the algorithm achieves an 80% detection probability even at an aggressive false positive rate threshold (5%).

These results reinforce the theoretical claim that the discriminative power of the approach directly couples to the informativeness of the examined states/segments, the volume of training data, and relative noise magnitudes.

Implications and Future Directions

The proposed method marks a significant methodological advancement in model-free anomaly detection for nonlinear dynamical systems, balancing the expressivity of GPs with rigorous uncertainty quantification. It avoids the pitfalls of black-box neural methods (e.g., Bayesian NNs) that lack statistical interpretability or make prohibitive data demands. The approach's modularity allows direct integration with existing statistical hypothesis testing paradigms.

From a practical standpoint, applications extend to quality control in manufacturing, online health monitoring, and adaptive control systems, where explicit models are unavailable but strict guarantees on false alarm rates are necessary.

The paper highlights several promising research directions:

• Optimal experiment design: Leveraging the GP-derived uncertainty, future work can optimize probe selection (e.g., initial state design) or adaptively select informative time intervals.
• Kernel engineering: Integrating structural priors via custom kernels or augmenting the nominal GP model with domain-specific inductive biases.
• State-dependent weighting: Dynamic residual weighting to target the detection of specific classes of anomalies, or deploy the method hierarchically in multimodal or hybrid systems.

Conclusion

This work establishes a statistically grounded, model-agnostic framework for anomaly detection in dynamical systems via GP regression. By deriving closed-form expressions for all sources of residual uncertainty and coupling them to a flexible, data-driven detection threshold, the paper advances both the theoretical and practical state of the art in uncertainty-aware system discrimination. These contributions set a foundation for future extensions in scalable testing, adaptive estimation, and domain-informed residual analysis.