Multi-Horizon Neural Surrogates

• The paper presents a novel approach where a single trained network predicts long-range states via continuous-horizon, recourse, and multi-stepsize operator surrogates.
• It integrates diverse methods such as FiLM-based horizon conditioning and MILP embedding to handle nonlinear dynamics, PDEs, and stochastic programming challenges.
• The framework achieves significant speedups and reduced error accumulation through effective error quantification and adaptive trust-regulation mechanisms.

Multi-horizon neural surrogates are neural architectures and training recipes explicitly designed to map initial conditions and parameterizations to future states or objective quantities across a range of temporal horizons in dynamical, stochastic, or decision-making problems. These models unify direct long-horizon state prediction, multi-step-ahead recourse estimation, and horizon-parameterized rollout stability in nonlinear dynamics, PDEs, and stochastic programs. Their central methodological advance is to offer high-fidelity, high-throughput forward simulation or operational cost approximation at multiple horizons using a single trained network, while controlling or quantifying error growth and predictive reliability.

1. Mathematical Formulations and Horizon Parameterization

Multi-horizon neural surrogate models fundamentally extend classic surrogate or operator learning, replacing the one-step or single-horizon mapping with parameterized flow maps or recourse-function surrogates that condition directly on the desired prediction horizon.

Continuous-horizon State Surrogates:

Define state space $\mathcal S$ and ground-truth flow $\Phi_T:\mathcal S \to \mathcal S$ induced by a reference solver advancing by horizon $T$. The surrogate learns

$f_\theta(s_0,T) \approx \Phi_T(s_0)$

with a supervised regression loss

$$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$

where $T$ is sampled from a geometric ladder and $\tilde{x}$ denotes per-channel normalization. This framework is architecture-agnostic and extends to U-Nets for PDE fields or residual MLPs for low-dimensional ODE states (Lakshmanan et al., 27 May 2026).

Multi-Horizon Recourse Surrogates for Stochastic Programming:

In multi-stage stochastic programming (MHSP), let $Q_i(x_i)$ be the expected recourse function at decision node $i$. Train a feed-forward ReLU network to approximate $Q_i(x_i)$:

$\Phi_T:\mathcal S \to \mathcal S$0

and embed $\Phi_T:\mathcal S \to \mathcal S$1 directly into the MILP, resulting in a mixed-integer program whose complexity scales with network size but not scenario count. The loss is

$\Phi_T:\mathcal S \to \mathcal S$2

yielding fast, scalable approximations for power-system and other operational planning problems (Zhang et al., 2 Dec 2025).

Multi-Stepsize Operator Surrogates:

Frame the surrogate as a family of step-size parameterized operators, $\Phi_T:\mathcal S \to \mathcal S$3, trained to predict the state after $\Phi_T:\mathcal S \to \mathcal S$4 strides given $\Phi_T:\mathcal S \to \mathcal S$5:

$\Phi_T:\mathcal S \to \mathcal S$6

with loss

$\Phi_T:\mathcal S \to \mathcal S$7

This parameterization makes horizon generalization explicit and supports both fine and coarse predictive timesteps in turbulence and flow problems (Pan et al., 14 Apr 2026).

2. Architecture Design and Horizon Conditioning

Modern multi-horizon surrogates incorporate architectural and embedding strategies for continuous or discrete horizon conditioning and multi-timescale specialization.

Horizon Conditioning:

• FiLM Embedding: Embed continuous horizon $\Phi_T:\mathcal S \to \mathcal S$8 with a small MLP, producing scale and shift parameters $\Phi_T:\mathcal S \to \mathcal S$9; modulate model activations via $T$0 at multiple depths (minimal architectural change for horizon-awareness) (Lakshmanan et al., 27 May 2026).
• Stride Routing/Mixture-of-Experts: In Multi-Stepsize Mixture-of-Experts (Ms-MoE) operators, a log-scaling router activates stride-specific and shared experts $T$1, with Gaussian weighting $T$2 and a stride-indexed corrector $T$3 (Pan et al., 14 Apr 2026).

Model Backbones:

• U-Nets (for spatial PDE states), MLPs (low-dim ODE), and advanced neural operators (implicit/multi-axis Transformer architectures).
• Incorporation of explicit temporal integrators—either via learned vector fields (NeuralODE), explicit residual dynamics (CoRD), or hybridized with classical ODE solvers for continuous-time flexibility (Zhou et al., 2024, Biswas, 24 May 2026).

MILP Embedding for Stochastic Programs:

• ReLU-activated feedforward networks reformulated as linear constraints for embedding into mathematical programs (Zhang et al., 2 Dec 2025).

Error-Quantification and Trust Regulation:

• Per-trajectory or per-cell error map: discrepancy between single-long and chained two-short horizon predictions, producing a trusted error heatmap that localizes discontinuities (Lakshmanan et al., 27 May 2026).

3. Training Protocols, Objective Strategies, and Closed-Loop Performance

Dataset Generation:

• Trajectory pairs $T$4 generated by textbook solvers for a range of sampled horizons.
• Data augmentation: e.g., stratified horizon selection (geometric ladders), controlled noise injection.

Loss Functions:

• Direct MSE on normalized state or outcome spaces.
• Composite losses for multi-fidelity or multi-task LSTM surrogates: joint or sequential training across fidelity levels (Conti et al., 2022).
• Denoising-score-matching in diffusion models for self-refinement across sequential noise levels (Liu et al., 18 Mar 2026).
• Multi-stride supervision for Ms-MoE: uniformly sample $T$5 pairs during training (Pan et al., 14 Apr 2026).

Closed-Loop Rollout Stability:

• Rollout diagnostics: finite-time Lyapunov exponents, local Jacobian spectra, per-step bias, invariant/statistical attractor comparison, regime-wise error bins (Biswas, 24 May 2026).
• Ablation studies confirm importance of residual updates, sub-stepping, and global conditioning for bias and error amplification control.

4. Trust, Uncertainty Quantification, and Hybrid Deployment

Trust Indicators and Error Localization:

• Step-doubling/consistency error: $T$6 defines an error map localizing shocks or contacts without explicit supervision (Lakshmanan et al., 27 May 2026).

Fallback Mechanisms and Hybrid Mode:

• Trust-aware fallback defers to a reference solver for uncertain predictions (error map above quantile $T$7), yielding a strict trade-off between throughput and risk: at $T$8 (25% deferral), residual surrogate error is halved while maintaining $T$9–$f_\theta(s_0,T) \approx \Phi_T(s_0)$0 speedup (Lakshmanan et al., 27 May 2026).

Benchmarking Against Uncertainty Baselines:

Multi-horizon surrogates with error localization outperform or match deep ensembles, learned error heads, conformal prediction, and classical step-size doubling, while relying on a single uncalibrated network (Lakshmanan et al., 27 May 2026).

5. Performance Benchmarks and Empirical Evaluations

Domain / Benchmark	Horizon / Scenario	Surrogate	Metric / Finding	Reference
Reaction-diffusion, Euler-2D, Ball-3D	$f_\theta(s_0,T) \approx \Phi_T(s_0)$1, batch size 1+	U-Net, MLP	$f_\theta(s_0,T) \approx \Phi_T(s_0)$2–$f_\theta(s_0,T) \approx \Phi_T(s_0)$3 CPU speedup vs solvers	(Lakshmanan et al., 27 May 2026)
Turbulence (HIT, channel)	Multi-stride ($f_\theta(s_0,T) \approx \Phi_T(s_0)$4)	Ms-MoE-IFactFormer	Stable rollouts, match DNS statistics $f_\theta(s_0,T) \approx \Phi_T(s_0)$5 steps	(Pan et al., 14 Apr 2026)
Chaotic ODEs/PDEs (KS, double pend.)	Rollout to blow-up	NeuralODE, CoRD	$f_\theta(s_0,T) \approx \Phi_T(s_0)$6–$f_\theta(s_0,T) \approx \Phi_T(s_0)$7 lower bias and FTLE	(Biswas, 24 May 2026)
Stochastic programming (UK power)	$f_\theta(s_0,T) \approx \Phi_T(s_0)$8 up to 50 scenarios	ReLU FFNN in MILP	Solve time speedup $f_\theta(s_0,T) \approx \Phi_T(s_0)$9–$$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$0, $$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$1	(Zhang et al., 2 Dec 2025)
Parametric time-dependent ODE/PDE	Full horizon $$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$2	Multi-fidelity LSTM	MF LSTM reduces MSE by $$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$3–$$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$4 orders vs single-fidelity	(Conti et al., 2022)

Multi-horizon surrogates systematically extend stable forecasting window and lower error accumulation compared to one-step or single-horizon surrogates.

6. Technical Limitations, Extensibility, and Open Directions

Known Limitations:

• Offline data/label generation remains a bottleneck in high-dimensional or multi-scenario contexts (Zhang et al., 2 Dec 2025).
• Model size and MILP complexity scale with neuron count in stochastic-programming surrogates (Zhang et al., 2 Dec 2025).
• Ms-MoE-IFactFormer adds memory/compute overhead ($$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$550% parameters, $$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$6 epoch time) (Pan et al., 14 Apr 2026).

Enhancements Under Investigation:

• Adaptive/active scenario sampling and curriculum for data efficiency in large scenario settings (Zhang et al., 2 Dec 2025).
• Multi-step or compositional loss enforcement for better horizon consistency in operator surrogates (Pan et al., 14 Apr 2026).
• Integration of continuous-time neural operators or neural ODEs for resolution-invariant rollouts and flexible timestep inference (Zhou et al., 2024, Biswas, 24 May 2026).
• Sparse regularization and expert pruning in mixture-of-experts architectures to control memory cost as $$\mathcal L_{\mathrm{sup}}(\theta) = \mathbb E_{(s_0,s_T,T)} \|\tilde{f}_\theta(s_0,T)-\tilde{s}_T\|_2^2$$7 grows (Pan et al., 14 Apr 2026).
• Adaptive trust/reject curves for dynamic accuracy-throughput trade-off in hybrid regimes (Lakshmanan et al., 27 May 2026).
• Uncertainty quantification and error-propagation theory in self-refining diffusion surrogates (Liu et al., 18 Mar 2026).

This domain remains at a critical intersection of numerically reliable modeling and extreme acceleration of scientific simulation, decision-making, and uncertainty analysis. The efficacy, flexibility, and technical soundness of multi-horizon neural surrogates are established across paradigms by direct algorithmic and experimental evidence (Lakshmanan et al., 27 May 2026, Zhang et al., 2 Dec 2025, Conti et al., 2022, Biswas, 24 May 2026, Liu et al., 18 Mar 2026, Zhou et al., 2024, Pan et al., 14 Apr 2026).