Robust Sparse Identification of Nonlinear Dynamics via Least Trimmed Squares

Abstract: In this work, we propose a robust Sparse Identification of Nonlinear Dynamics (SINDy) pipeline for handling datasets corrupted by noise and outliers. The method decouples outlier filtering from sparse regression by combining Iterative Least Trimmed Squares (ILTS) with Sequentially Thresholded Least Squares (STLS). Unlike standard approaches that treat all observations uniformly within a single regression stage, the proposed ILTS-SINDy framework first applies an ILTS procedure that iteratively minimizes the sum of the smallest squared residuals to identify the most reliable observations without prior knowledge of outliers, after which STLS is used to recover a parsimonious governing model. Extensive numerical experiments show that ILTS-SINDy can significantly outperform existing robust SINDy variants across a range of outlier contamination levels, with performance maintained even under settings with up to $20\%$ corrupted observations.

• The paper introduces ILTS-SINDy, a filter-then-sparsify method that uses iterative least trimmed squares to remove corrupted observations before applying sequential thresholded least squares.
• ILTS-SINDy achieved the strongest support recovery across SIR, Lorenz, and Lotka–Volterra benchmarks, retaining near-perfect accuracy at up to 20% outliers and 20% noise.
• The results show that filtering improves exact equation recovery more reliably than ensemble or joint approaches, although the method currently requires the outlier fraction and relies on a central-difference-specific trimming heuristic.

Motivation and contribution

The Sparse Identification of Nonlinear Dynamics (SINDy) framework recovers governing ODEs by regressing finite-difference derivative estimates against a library of candidate functions, then enforcing sparsity. Its principal weakness is sensitivity to corrupted data: a single outlier contaminates not only the regression matrix row but also up to three central-difference derivative estimates, and this error propagates into spurious model terms. Existing robustifications each carry drawbacks — Weak SINDy requires careful test-function selection, Ensemble SINDy is computationally costly and inherits biases from corrupted resamples, and trimmed SINDy variants couple outlier selection and sparse regression in a single joint optimization, so heavily corrupted samples may still be classified as inliers.

The paper proposes ILTS-SINDy, a filter-then-sparsify pipeline that decouples these two tasks. An Iterative Least Trimmed Squares (ILTS) procedure first identifies a trusted subset of observations by alternating between least-squares fitting on the current inlier set and re-selection of the $p$ smallest-residual samples; Sequentially Thresholded Least Squares (STLS) then recovers sparse coefficients from the cleaned data pair $(\hat{A}, \hat{b}_i)$. This corresponds to an approximate solution of the non-convex Least Trimmed Squares objective $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$, i.e., minimization over the $p$ smallest squared residuals.

Method

The pipeline per state variable $x_i$ proceeds as follows:

1. Initialization: compute the ordinary least-squares estimate on all data.
2. ILTS trimming: iteratively select the $p$ observations with smallest residuals, refit restricted least squares on that subset, and terminate when the trimmed residual sum ceases to decrease. The output includes the cleaned matrix $\hat{A}\in\mathbb{R}^{p\times d}$ and target vector $\hat{b}_i\in\mathbb{R}^p$.
3. STLS sparsification: apply hard thresholding with parameter $\lambda$ and restricted least-squares refits until the active set stabilizes.

A key design choice is the inlier count $p = (m+1) - \lfloor 3q(m+1)\rfloor$, where $(\hat{A}, \hat{b}_i)$0 is the assumed outlier fraction. The multiplier $(\hat{A}, \hat{b}_i)$1 compensates for stencil contamination: one corrupted snapshot corrupts its own and two neighboring derivative estimates. The appendix ablation justifies this choice — recall of true outliers peaks near $(\hat{A}, \hat{b}_i)$2 across all three benchmarks (e.g., $(\hat{A}, \hat{b}_i)$3 for SIR at $(\hat{A}, \hat{b}_i)$4 versus $(\hat{A}, \hat{b}_i)$5 at $(\hat{A}, \hat{b}_i)$6), approaching the theoretical "exact" values ($(\hat{A}, \hat{b}_i)$7, $(\hat{A}, \hat{b}_i)$8, $(\hat{A}, \hat{b}_i)$9). Notably, recall behaves non-monotonically in $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$0, and the optimum shifts across noise realizations; $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$1 is adopted as a system-agnostic heuristic rather than a provably optimal setting.

Experimental setup

Benchmarks are the SIR, Lorenz, and Lotka–Volterra systems, integrated via RK4 with polynomial candidate libraries (quadratic without self-interactions for SIR; cubic for Lorenz and Lotka–Volterra). Corruption is injected as additive Gaussian noise scaled by each state's RMS value, applied to a randomly selected fraction $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$2 of snapshots. For each of 64 pairs $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$3 with both ranging over $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$4, 100 independent trials are run — 6,400 trajectories per model — evaluated by mean support recovery accuracy and exact recovery rate. Comparisons are against standard STLS-based SINDy and E-SINDy (PySINDy defaults).

Results

Two findings stand out. First, ILTS-SINDy achieves the highest mean support recovery accuracy across all systems and corruption levels, remaining close to 1 even at 20% outliers and 20% noise, whereas standard SINDy degrades sharply, particularly on SIR and Lotka–Volterra. Second — and more diagnostic — the exact recovery results separate the methods decisively: E-SINDy often attains high mean accuracy while its exact identification rates remain low across most settings, showing that accurate trajectory-level prediction does not imply correct structure recovery. ILTS-SINDy maintains near-perfect exact recovery on SIR and strong performance on Lorenz and Lotka–Volterra even under severe contamination. Noise degrades performance more than outliers for all methods, consistent with the fact that noise perturbs every derivative estimate while outliers affect only localized stencils that trimming can excise.

The practical implication is that decoupling filtering from sparsification yields structurally correct models where joint or ensemble approaches recover only approximately correct supports — a distinction that matters when the identified equations are used for interpretation or extrapolation rather than short-horizon prediction.

Limitations and open questions

The authors state plainly that the headline comparison assumes exact knowledge of the outlier percentage $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$5, which is rarely available in practice; the adaptive estimation of $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$6 (and hence $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$7) is deferred to future work. The multiplier $$\min_{\alpha_i} \frac{1}{2}\sum_{j=0}^{p-1} r_{\ell_j}(\alpha_i)^2$$8 is an empirical heuristic tied specifically to three-point central differences, and its transfer to higher-order or integral differentiation schemes is unexamined. Evaluation is confined to synthetic benchmarks with Gaussian corruption; no real-world datasets are tested, and behavior under non-Gaussian or correlated noise is not characterized. Extension to SINDy with control inputs is also left open.

Conclusion

ILTS-SINDy is a simple two-stage modification of the standard SINDy pipeline — ILTS-based trimming followed by STLS — that substantially improves exact recovery of governing equations under combined noise and outlier contamination, outperforming standard SINDy and E-SINDy across three benchmark systems at corruption levels up to 20%. Its main unresolved dependency is the prior specification of the outlier fraction, which currently must be supplied by the user.