VAR+FNN Residual Correction Framework

• The paper introduces a VAR+FNN framework that couples linear VAR predictions with neural network corrections to achieve significant order-of-magnitude error reductions.
• The hybrid methodology uses a two-stage approach where the VAR model captures dominant dynamics and the FNN models nonlinear residuals, enhancing model accuracy.
• The framework’s theoretical error analysis and empirical performance in finance and PDE surrogate modeling demonstrate its robustness, efficiency, and practical impact.

The residual correction framework, often operationalized as a coupled Vector AutoRegressive (VAR) plus Feedforward Neural Network (FNN) pipeline (here, "VAR+FNN"), is a hybrid modeling approach in which a well-understood linear system captures dominant dynamics and a neural network is trained to model the residual—or error—between the linear prediction and observed outcomes. This two-stage method has wide applicability, including temporal forecasting in econometrics, high-frequency trading, physics-informed surrogates for nonlinear PDEs, and scientific computing. The VAR+FNN paradigm inherits interpretability from classical statistical models while exploiting the universality and nonlinearity-capturing power of neural networks. The framework and its variants support both offline and online learning and admit rigorous theoretical analysis, including a priori error estimates and quantitative demonstrations of computational efficiency.

1. Mathematical Foundation and Model Architecture

The VAR+FNN framework relies on a hierarchical decomposition of the prediction task. Consider a $k$-dimensional time series $\mathbf Y_t$:

$$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$

where $\mathbf c$ is a vector intercept, $\mathbf A_i$ are the lag matrices, and $\boldsymbol\varepsilon_t$ are innovation residuals. The VAR fit is computed using ordinary least squares, selecting the lag $p$ via information criteria (BIC/AIC).

The innovation (residual) at time $t$ is then

$$\boldsymbol\varepsilon_t = \mathbf Y_t - \widehat{\mathbf Y}_t^{\mathrm{VAR}} = \mathbf Y_t - \left( \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} \right).$$

A feedforward neural network $f_\Theta : \mathbb R^k \to \mathbb R^k$ models nonlinear structure in $\mathbf Y_t$0: $\mathbf Y_t$1

For scalar response or higher-dimensional mapping, subspace or coordinate-wise corrections are feasible.

In the context of surrogate modeling for PDE solutions, the analogous framework operates by

1. Producing an FNN-based prediction $\mathbf Y_t$2 for a PDE parameter $\mathbf Y_t$3;
2. Assembling a problem-specific residual operator (often variational, e.g., Newton linearization);
3. Solving a correction linear system to update the FNN output via $\mathbf Y_t$4 (Jha, 2023, Cao et al., 2022).

2. Theoretical Error Analysis and Justification

The efficacy of the two-stage residual correction relies on key properties of the underlying systems. For nonlinear variational problems, the use of a single Newton-type correction can provably reduce the approximation error of the FNN surrogate from $\mathbf Y_t$5 to $\mathbf Y_t$6 under Newton–Kantorovich regularity hypotheses (Jha, 2023, Cao et al., 2022). Specifically:

$\mathbf Y_t$7

In time-series applications, the FNN functions as a universal approximator for the nonlinear part, with the linear VAR conferring structure and interpretability. The hybrid reduces estimation bias and variance and, in empirical applications, achieves order-of-magnitude error reductions, as demonstrated in high-frequency trading order-flow forecasting (Rahman et al., 2024).

The "Universal Differential Equation" philosophy undergirds this approach: analytically tractable or linearizable physics/statistics models are augmented by minimal nonlinear corrections, focusing limited neural capacity on essential residual dynamics (Chen et al., 1 Dec 2025).

3. Algorithmic Workflow for Residual Correction

The algorithmic pipeline for the VAR+FNN framework—abstracting over application domains—follows:

1. Fit the linear baseline model (VAR or analytic/surrogate physics operator); compute residuals on observed or synthetic data.
2. Train an FNN to model the relationship $\mathbf Y_t$8 nonlinear correction, typically using mean squared error on the residuals as the loss.
3. Deploy the hybrid model, combining the baseline prediction and learned correction as

$\mathbf Y_t$9

1. Optionally, for PDE-based tasks, after obtaining the correction increment via a linear system solve, add it to the FNN output.

In online data assimilation (4D-Var), a small FNN is inserted to correct the model tendencies, and its parameters are updated "online" within the inner loop of incremental variational assimilation, using adjoint-based gradients and sequential outer–inner optimization (Farchi et al., 2022).

4. Integration with Applications and Empirical Performance

In econometrics and financial engineering, the hybrid VAR+FNN method has demonstrated substantial improvements in prediction of order-flow imbalance (OFI). For example, on BTCUSD and ETCUSDT datasets (3,000 samples each), the hybrid model reduced the mean squared error (MSE) from 0.021 (FNN-only) and 0.675 (VAR-only) to 0.002, and increased $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$0 to 0.997. Classification accuracy of trading signals derived from OFI exceeded 98%, far above the individual model baselines (Rahman et al., 2024).

For neural operator surrogates in nonlinear PDEs, the corrector framework achieved accuracy gains of two orders of magnitude in percent error (dropping from $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$110% to $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$2) and sharply reduced topology optimization errors from 80% to below 7% with minimal additional computational cost (equivalent to a single FE linear solve) (Jha, 2023).

Bayesian inverse problems benefit from the correction step by attaining near-perfect $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$3-accuracy without the computational burden of iterative PDE solves. Speedups of order $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$4–$$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$5 compared to direct Newton/PDE solution are typical, with negligible sacrifice in posterior quality, and a one-step correction yielding nearly all available accuracy enhancement (Cao et al., 2022).

5. Architectural Variants and Design Considerations

The FNN residual corrector is typically of modest size: 2–3 layers of 16–32 units for geophysical and econometric applications, with ReLU or tanh/swish activations (Rahman et al., 2024, Farchi et al., 2022). In physics applications (e.g., "NNPT"), a compact MLP (e.g., $$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$6) suffices to learn residuals induced by perturbations about an analytic solution—achieving 28–54$$\mathbf Y_t = \mathbf c + \sum_{i=1}^p \mathbf A_i \mathbf Y_{t-i} + \boldsymbol\varepsilon_t,$$7 lower validation MSE versus end-to-end fitting of the full signal. Notably, there are sharp architecture-related capacity transitions at physical complexity thresholds, such as chaotic onset in the three-body problem (Chen et al., 1 Dec 2025).

For high-dimensional inputs (e.g., spatial PDE parameter fields), low-rank representations using SVD/PCA are often exploited prior to application of the FNN. For example, "PCANet" structures embed dimension reduction and expansion around a stack of ResNet blocks (Jha, 2023). For online 4D-Var model error correction, column-wise subnetworks with standardized input/output channels are common (Farchi et al., 2022).

6. Practical Impact and Limitations

The VAR+FNN paradigm provides a statistically robust and computationally efficient strategy to correct structured model errors, yielding state-of-the-art surrogate accuracy in diverse domains. For PDE surrogates, it enables black-box correction and provable quadratic error reduction with single-step Newton updates. In financial forecasting, it delivers near-perfect accuracy for key portfolio variables and enhances high-frequency trading strategies.

Limitations include potential failure in regimes of extremely poor baseline approximation, insufficient FNN capacity relative to system complexity, and cases where the residual structure does not admit efficient decoupled modeling. The pragmatic choice of FNN size, depth, and normalization, as well as online versus offline training, remains problem-dependent and requires empirical tuning.

7. Generalizations and Connections to Broader Hybrid Modeling

The residual correction architecture instantiated by VAR+FNN (or analytic propagator + FNN) aligns with broader correction-learning strategies in scientific machine learning. In particular, Neural Network Perturbation Theory (NNPT) applies the same philosophy to dynamical systems by learning only deviations from a trusted analytic baseline, focusing neural capacity on essential corrections and revealing "capacity barriers" at the onset of chaos (Chen et al., 1 Dec 2025).

This approach admits variational generalizations, online adaptation, and integration with data assimilation schemes such as incremental 4D-Var. The decoupling of linear/analytic and nonlinear components, the use of FNNs as universal residual correctors, and the algorithm’s ease of integration into existing simulation or econometric pipelines illustrate the flexibility of the VAR+FNN residual correction framework across the computational sciences.