Deep Predictor-Corrector Networks for Robust Parameter Estimation in Non-autonomous System with Discontinuous Inputs

Abstract: Learning under non-smooth objectives remains a fundamental challenge in machine learning, as abrupt changes in conditioning variables can induce highly non-smooth loss landscapes and destabilize optimization. This difficulty is particularly pronounced in non-autonomous dynamical systems driven by discontinuous inputs, where widely used optimization methods, including recent neural smoothing approaches, exhibit unreliable convergence or strong hyperparameter sensitivity. To address this issue, we propose Deep Predictor-Corrector Networks (DePCoN), a multi-scale learning framework that stabilizes optimization by learning scale-consistent parameter update rules across a hierarchy of smoothed inputs. Rather than treating smoothing as a fixed preprocessing choice, DePCoN integrates smoothing into the learning dynamics itself through a learned predictor-corrector mechanism. Across biological and ecological benchmarks with discontinuous inputs, DePCoN consistently achieves more robust and faster convergence than existing methods while substantially reducing sensitivity to hyperparameter choices. Beyond dynamical systems, our approach provides a general learning principle for stabilizing optimization under non-smooth objectives.

• The paper introduces the DePCoN framework that uses multi-scale smoothing and neural ODE-based correction to robustly recover parameters from systems with abrupt inputs.
• The methodology integrates a predictor network to propagate estimates and a corrector stage for data consistency, significantly reducing hyperparameter sensitivity.
• Experimental results on benchmarks, including circadian pacemaker and Lotka–Volterra models, demonstrate superior convergence, stability, and accuracy.

Deep Predictor-Corrector Networks for Robust Parameter Estimation in Non-autonomous Systems with Discontinuous Inputs

Motivation and Context

Parameter estimation for non-autonomous dynamical systems with discontinuous exogenous inputs is notoriously ill-posed due to loss landscape irregularities induced by abrupt input shifts. Such phenomena are ubiquitous in practice, occurring across domains such as biological rhythms, ecological systems, control theory, power electronics, and financial modeling. Existing optimization algorithms—both classical and neural-based smoothing approaches—struggle with convergence reliability and hyperparameter sensitivity. Specifically, methods that apply kernel-based or neural smoothing treat the smoothing scale as a fragile hyperparameter, trading optimization stability against estimation bias and undermining practical robustness.

Limitations of Existing Approaches

Conventional optimization algorithms (e.g., L-BFGS, LM, DE, NM, SLSQP) and recent neural smoothers like HADES-NN generally fail to consistently recover parameters when confronted with real-world discontinuous inputs. Empirical analysis reveals large estimate dispersion and sensitivity to random initialization, as depicted by scatter plots for the human circadian pacemaker model (Figure 1), indicating unreliable convergence in regimes of irregular light exposure.

Figure 1: Discontinuous inputs generate highly non-smooth loss surfaces, causing instability and hyperparameter sensitivity in HADES-NN and classical optimizers.

Neural smoothing methods (e.g., HADES-NN) improve estimator accuracy to an extent but remain acutely sensitive to smoothing depth ($M$). Insufficient smoothing perpetuates instability; excessive smoothing suppresses informative signal variations and induces bias (Figure 1d). Computational cost also scales unfavorably with smoothing iterations.

The DePCoN Framework: Multi-scale Predictor–Corrector Architecture

To address these pathological optimization dynamics, Deep Predictor–Corrector Networks (DePCoN) are introduced as a principled multi-scale parameter estimation framework. The architecture leverages a hierarchy of smoothed input signals, obtained via heat-kernel convolution with varying scales, integrating smoothing directly into the learning dynamics. The framework consists of a predictor stage that propagates parameter estimates from coarse to fine smoothing scales, and a corrector stage that enforces data consistency using Neural ODE-based system identification at each scale. Training is performed by minimizing a multi-scale loss aggregating discrepancies across all smoothing levels, thus transforming smoothing from a fragile hyperparameter into a structured learning input.

Figure 2: DePCoN utilizes multi-scale smoothing, a predictor network propagating parameters, and a corrector enforcing consistency across all scales.

The predictor network is implemented as a feed-forward FCNN with ReLU activations, sequentially mapping parameter estimates between smoothing levels. The corrector network evaluates predicted parameters by integrating the smoothed-system dynamics using Neural ODE solvers and compares to observation data. End-to-end learning aggregates errors across hierarchically smoothed inputs, enforcing scale-consistent parameter recovery and minimizing sensitivity to any single smoothing choice.

Robustness, Efficiency, and Hyperparameter Insensitivity

DePCoN demonstrates marked robustness to hyperparameter settings. Unlike HADES-NN, whose accuracy and computational trajectory are sensitive to smoothing depth and network configuration, DePCoN yields tightly concentrated parameter estimates regardless of smoothing-grid size ($N$), as shown by the distribution of estimates for the circadian pacemaker model (Figure 3a) and stable MAPE box plots (Figure 6a). DePCoN also exhibits superior convergence rates and time-to-accuracy profiles relative to HADES-NN, efficiently entering low-error regimes without protracted optimization (Figure 3b).

Figure 3: DePCoN delivers hyperparameter robust parameter estimation and accelerated convergence compared to HADES-NN.

Figure 4: MAPE distributions reveal DePCoN’s hyperparameter-invariant accuracy across smoothing-grid sizes for both benchmark systems.

Generalization and Benchmarking

Robustness extends to ecological benchmarks, e.g., modified Lotka–Volterra systems under abrupt environmental forcing. Classical optimizers and HADES-NN, even under favorable hyperparameter settings, show large dispersion and systematic bias (Figure 4b-c). DePCoN is unique in stably recovering tightly clustered parameter estimates at the ground truth across random initializations (Figure 4d). MAPE values remain insensitive to smoothing-grid variations (Figure 6b), confirming practical robustness.

Figure 5: Classical optimizers and HADES-NN fail under discontinuous inputs; DePCoN accurately and consistently recovers LV parameters.

DePCoN outperforms all baselines in quantitative accuracy and stability, as evidenced by MAPE and per-parameter statistics for both circadian and Lotka–Volterra benchmarks, with standard deviations less than an order of magnitude smaller than competing approaches.

Theoretical Analysis and Convergence Guarantees

The paper establishes rigorous convergence analysis: under standard regularity and identifiability assumptions, the sequence of optimal parameter estimates for each smoothing scale converges to the ground-truth minimizer as smoothing vanishes. This theoretical foundation validates smoothing-based continuation and the multi-scale loss approach, ensuring that DePCoN’s learned parameters remain close to the sequence of loss minimizers.

Practical and Theoretical Implications

DePCoN generalizes beyond ODE parameter estimation, suggesting a broad learning principle for stabilizing optimization under non-smooth objectives. Potential applications include hybrid dynamics, machine learning models trained under discontinuous conditioning variables, and system identification tasks in control, ecology, neuroscience, and epidemiology. By embedding smoothing into the estimation process and enforcing scale-consistent learning, DePCoN transforms a historically brittle hyperparameter into a robust architectural feature, yielding practical reliability and systematic accuracy.

Comparison with Prior Work and Future Directions

The predictor-corrector concept aligns with recent continuation and homotopy optimization heuristics, but DePCoN distinctively integrates smoothing into learnable dynamics across multiple scales rather than a single fixed auxiliary step. The approach provides a template for robust estimation under challenging loss landscapes, and future research may explore extension to nonparametric system identification and settings with implicit loss discontinuities (e.g., reinforcement learning with abrupt reward structures).

Figure 6: Only DePCoN consistently recovers ground-truth circadian parameters across trials, outperforming both neural and classical baselines.

Conclusion

DePCoN achieves robust, accurate, and hyperparameter-invariant parameter estimation for non-autonomous systems with discontinuous exogenous inputs, outperforming established optimizers and neural smoothing baselines in both stability and computational efficiency. The multi-scale predictor-corrector architecture introduces structured input regularization as an intrinsic component of the learning process, providing a general route to stabilization in nonsmooth optimization settings. This methodological advance has broad theoretical and practical implications for machine learning involving discontinuous signals and irregular loss landscapes (2603.12965).