Adaptive NVAR with MLP: Scalable Forecasting

• Adaptive NVAR is a hybrid model that combines delay-based linear autoregression with a jointly trained shallow MLP to capture nonlinear time-series dynamics.
• Its methodology integrates delay-embedded features with dropout-regularized MLP mappings, enhancing robustness against noise and improving scalability in high-dimensional settings.
• Empirical results demonstrate significant RMSE reductions in chaotic systems and SST forecasting, validating its accuracy and practical application in diverse domains.

Adaptive Nonlinear Vector Autoregression (NVAR) with Multi-Layer Perceptron (MLP) augmentation represents a principled synthesis of time-delay linear autoregressive modeling and shallow neural network-driven nonlinear feature learning. Emerging from limitations in classical NVAR and reservoir computing—primarily the use of fixed nonlinear mappings and hindered scalability—adaptive NVAR unifies the interpretable lag structure of vector autoregression with the expressive capacity and regularization advantages of shallow neural architectures. This approach yields robust, scalable, and accurate forecasting in chaotic, noisy, or high-dimensional dynamical systems, and exhibits improved performance in both synthetic and real-world domains such as oceanographic field forecasting and macroeconomic prediction (Sherkhon et al., 11 Jul 2025, Azimov et al., 10 Jun 2026, Cabanilla et al., 2019).

1. Architectural Principles and Model Formulation

Adaptive NVAR departs from canonical NVAR by replacing static polynomial or random-feature expansions with a shallow, jointly trained MLP that serves as a nonlinear, data-adaptive mapping. Formally, given a $d$-dimensional time series $\{x_t\}_{t=1}^T$, the model first constructs a delay-embedded linear input:

$$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$

for lag order $k$ (typically $k=30$ for chaotic tasks). The MLP block then learns

$$H_{NN,t} = W \cdot \mathrm{dropout}(\tanh(W_{in}z_t+b_1)) + b_2\,,\quad H_{NN,t}\in\mathbb{R}^m$$

where $W_{in}\in\mathbb{R}^{h\times (dk)}$, $W\in\mathbb{R}^{m\times h}$, $h$ and $m$ are chosen per task (e.g., $\{x_t\}_{t=1}^T$0, $\{x_t\}_{t=1}^T$1 for chaotic Lorenz systems). In noisy settings, dropout (e.g., $\{x_t\}_{t=1}^T$2) regularizes the nonlinear features; otherwise, it is omitted (Sherkhon et al., 11 Jul 2025, Azimov et al., 10 Jun 2026).

The core innovation lies in concatenating the original delay features and the nonlinear MLP features, then predicting the state increment with a single linear readout:

$\{x_t\}_{t=1}^T$3

where $\{x_t\}_{t=1}^T$4, and no explicit bias is used.

2. Training Methodology and Optimization

Adaptive NVAR parameters, comprising both the MLP ($\{x_t\}_{t=1}^T$5) and readout ($\{x_t\}_{t=1}^T$6), are trained end-to-end using mean squared error (MSE) on the one-step increment:

$\{x_t\}_{t=1}^T$7

with $\{x_t\}_{t=1}^T$8 and $\{x_t\}_{t=1}^T$9 a warm-up index. The optimization pipeline employs a two-phase regime: (i) Adam, with learning rate $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$0 for $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$1 epochs, establishes a robust initialization, followed by (ii) L-BFGS fine-tuning for precision convergence. Dropout substitutes for explicit ridge regularization, and early stopping or validation splitting addresses overfitting (Sherkhon et al., 11 Jul 2025, Azimov et al., 10 Jun 2026).

The multi-step forecasting procedure reuses the trained mappings, recursively updating the delay buffer and concatenated features at each forecast horizon.

3. Integration with Dimension Reduction: PCA/SVD-Enhanced Adaptive NVAR

In high-dimensional spatiotemporal domains, such as sea surface temperature (SST) field forecasting, adaptive NVAR is further enhanced by low-rank representation techniques. Full SVD decomposes the data matrix $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$2 ($$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$3: spatial grid points),

$$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$4

and the leading $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$5 empirical orthogonal functions (EOFs) are retained for near-lossless compression (e.g., $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$6 captures $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$7 variance in SST). The adaptive NVAR then models the delay-embedded evolution of the principal component time series $$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$8, with predictions in latent space mapped back to physical fields via

$$z_t = x_t \oplus x_{t-1} \oplus \dots \oplus x_{t-(k-1)} \in \mathbb{R}^{dk}$$9

(Azimov et al., 10 Jun 2026). All autoregressive embedding, MLP feature extraction, and training operate within the reduced-dimensional latent space, yielding significant acceleration and improved stability in large-scale applications.

4. Empirical Performance and Comparative Results

Experimental results on paradigmatic chaotic systems (e.g., Lorenz-63) and oceanographic SST fields underscore the adaptive NVAR's superior accuracy and robustness across a spectrum of noise regimes and forecast horizons:

• Lorenz-63, $k$0-step horizon, $k$1 Gaussian noise: adaptive NVAR achieves RMSE $k$2 versus $k$3 for standard NVAR ($k$4 reduction) (Sherkhon et al., 11 Jul 2025).
• East Sea SST, 90-day forecast: adaptive NVAR achieves RMSE $k$5 compared to $k$6 for standard NVAR (25% improvement) (Azimov et al., 10 Jun 2026).
• At lower observation frequencies (subsampling), adaptive NVAR maintains stable error, whereas standard NVAR exhibits dramatic degradation.

The following table summarizes forecasting results from (Sherkhon et al., 11 Jul 2025) (Lorenz-63, $k$7-component, RMSE $k$8 std):

Noise	H=25	H=50	H=75	H=100
0 %	0.007±0.004 vs 0.010	0.017±0.011 vs 0.035	0.039±0.046 vs 0.107	0.080±0.116 vs 0.235
5 %	0.192±0.035 vs 0.607	0.202±0.041 vs 1.166	0.278±0.078 vs 1.614	0.457±0.195 vs 3.876
10 %	0.413±0.050 vs 1.109	0.416±0.068 vs 2.254	0.492±0.154 vs 3.074	0.821±0.529 vs 7.925
15 %	0.654±0.103 vs 1.746	0.663±0.118 vs 3.833	0.774±0.277 vs 5.512	1.463±1.549 vs 9.244

For SST field prediction, adaptive NVAR achieves error reduction across all tested horizons, reflecting both short-term fidelity and long-horizon reliability (Azimov et al., 10 Jun 2026).

5. Connections to Related Neural VAR Architectures

Adaptive NVAR shares conceptual symmetry with neural enhancements to VAR, such as the vector autoencoder nonlinear autoregression (VANAR) (Cabanilla et al., 2019), where lag-embedded temporal features are processed by wide, shallow MLPs and, where beneficial, by encoder–decoder autoencoder modules. While VANAR also delivers forecasting and Granger-causality advantages over linear VAR, the adaptive NVAR adopts a more archetypal autoregressive delay stack and jointly trained shallow network, focusing on both scalability and interpretability. Both frameworks supersede the limitations of fixed polynomial NVAR expansions and conventional kernel-based methods in accommodating nonlinearity.

Further, recent works on interpretable nonlinear VARs have introduced invertible shallow MLP mappings per component (Roy et al., 2023), explicitly modeling monosensor nonlinearities atop latent VAR processes. These approaches, while still linear in the autoregressive backbone, emphasize interpretability, parameter sparsity, and tractable optimization—whereas adaptive NVAR targets higher aggregate nonlinear expressivity through the delay-MLP concatenation without enforcing invertibility.

Finally, attention-based time-series models have been rigorously aligned with dynamic VAR recurrences; SAMoVAR (Lu et al., 11 Feb 2025) exemplifies the integration of lightweight MLPs within a multi-layered, autoregressive Transformer block, delivering parameter efficiency, explicit lagwise interpretability, and SOTA forecast performance. The architectural convergence underscores that combining explicit autoregressive structure with learned nonlinear mappings—via MLPs or attention—yields predictively and interpretively robust time-series models.

6. Model Complexity, Scalability, and Practical Considerations

Adaptive NVAR's readout training avoids the matrix inversion costs of traditional NVAR and supports training and inference at scale due to joint gradient-based optimization. The replacement of hyperparameter-intensive grid search for feature expansion parameters with neural network hyperparameter-tuning (e.g., dropout rate, hidden dimension) simplifies application to new domains (Sherkhon et al., 11 Jul 2025). In high-dimensional contexts, dimensionality reduction (PCA/SVD) further accelerates computation without marked loss of predictive content (Azimov et al., 10 Jun 2026).

Early-stopping, standard weight decay, and, where needed, dropout and validation splitting are the principal regularization strategies. No explicit Tikhonov regularization is required, as neural components absorb variance in noisy settings.

7. Outlook and Research Directions

Adaptive NVAR establishes an effective paradigm for robust time-series forecasting under realistic conditions of high noise, nonlinear dynamics, and observational sparsity. Ongoing research addresses and explores:

• Extension to multivariate and gridded spatiotemporal domains via adaptive modeling in SVD or EOF subspaces (Azimov et al., 10 Jun 2026).
• Modular integration with recent autoregressive architectures (e.g., aligned Transformer-MLP hybrids (Lu et al., 11 Feb 2025)).
• Interpretability enhancements via sparsity constraints on either delay features or MLP weights.
• Systematic benchmarking against kernelized and nonlinear VAR models, especially in domains such as finance, climate, and sensor analytics (Roy et al., 2023).

As a general pattern, unifying interpretable autoregressive temporal structure with data-driven neural nonlinearities is emerging as a scalable and effective approach for forecasting, system identification, and causal analysis in complex dynamical settings.