Discretization Error Variance

• Discretization error variance is a measure quantifying uncertainty from approximating continuous models using discrete methods like time-stepping or spatial meshing.
• It arises in both deterministic error analysis and probabilistic frameworks, linking classical convergence rates with Bayesian uncertainty quantification.
• Practical applications include adaptive grid refinement, error-aware parameter estimation, and model validation in numerical PDEs and neural network surrogates.

Discretization error variance quantifies the uncertainty or stochastic dispersion introduced by replacing infinite- or continuum-domain mathematical models with finite, discrete representations—be it time-stepping, spatial meshing, or basis truncation. It is crucial as both a theoretical and practical metric in computational science, machine learning, statistical inference for inverse problems, and uncertainty quantification frameworks. The variance formulation arises in both deterministic error analyses and modern probabilistic approaches that treat discretization error itself as a random variable—either explicitly or as an asymptotic surrogate.

1. Formal Definitions and Classical Convergence Rates

Let $u$ denote the exact solution of a continuous model and $u_h$ its discrete counterpart under mesh, step-size, or Fourier truncation parameter $h$ (or $N$ in grid-based settings).

• The discretization error is defined as

$\delta_h := u_h - u,$

with the discretization error variance scaling algebraically as

$$\mathrm{Var}(\delta_h) = \mathbb{E}[|\delta_h|^2] \propto h^{2p}$$

for schemes of strong order $p$ or $N^{-2s}$ for Fourier-based discretizations with Sobolev regularity $s$ (Jourdain et al., 2017, Lanthaler et al., 3 May 2024, Marumo et al., 2023, Aalto, 2014).

• In operator learning and neural PDE surrogates, e.g. Fourier Neural Operators (FNO), the dominant source is “aliasing error,” arising from high-frequency mode folding in FFT-based grid evaluations. The variance of the layerwise error decays as $O(N^{-2s})$, with $N$ the grid size and $s$ the input regularity (Lanthaler et al., 3 May 2024).
• In SDE/time-stepping contexts, Euler–Maruyama or similar schemes yield discretization error with variance $O(h^2)$ for each step and accumulated mean-square error scaling as $O(h)$ for the whole trajectory (Choi et al., 10 Jun 2025, Jourdain et al., 2017).

2. Variance Quantification in ODE, PDE, and Filtering Models

Discretization error variance is essential in quantifying reliability and uncertainty in statistical inference, inverse problems, filtering algorithms, and numerical PDE solvers.

• Bayesian Discretization Error Models (ODEs):

Statistical models for parameter inference in ODEs treat solver error as additive Gaussian noise, with variance $\Sigma_k$ (potentially time- and state-dependent), estimated jointly with parameters via iteratively reweighted least-squares (IRLS) or advanced particle filtering (Matsuda et al., 2019, Toyota et al., 28 Nov 2025). The variance sequence $\{\sigma_k^2\}$ is often constrained to be monotonic (reflecting error accumulation) and consistent with the integration method’s convergence rate.

• Kalman Filtering in Infinite-dimensional Systems:

Projection of an infinite-dimensional state space onto a finite mesh (e.g., FEM) induces discretization error whose covariance evolves via a Riccati equation, with explicit operator-norm bounds. The steady-state variance scales as $h^{2\alpha}$, where $\alpha$ depends on the regularity of trial space and mesh size (Aalto, 2014).

• Stochastic Iterated Integrals:

For semimartingale-driven integrals, the variance of (appropriately normalized) discretization error converges to

$$V(t) = \frac{c^3}{6}\int_0^t \sigma_s^4 ds + \frac{41}{30} c^2 \sum_{0<s \le t} (\Delta Y_s)^2$$

where $c$ is the continuous martingale coefficient and $\sigma$ the local volatility (Song et al., 2017).

• Diffusion Model Discretization:

In generative modeling (DDPM), the mean-square error between exact and discretized SDE solution scales as $O(1/T)$ for $T$ time steps, i.e., variance $O(T^{-1})$ (Choi et al., 10 Jun 2025).

• Finite Element Methods:

Bayesian modeling approaches assign a Gaussian process prior to the solution and update the posterior covariance conditioned on discrete solution observations. The resulting posterior covariance reflects the dispersion due to mesh coarsening. Eigenvalue-rescaling may be required to achieve force-dependent error variance (Poot et al., 2023).

3. Probabilistic and Bayesian Frameworks for Error Variance

Recent methodologies interpret discretization error variance as an explicit random variable within Bayesian inference schemes:

• Wishart-based Covariance Estimation:

The block sample covariance of numerical error increments is modeled as Wishart-distributed, enforcing monotonicity via Löwner ordering and capturing off-diagonal correlations. The estimated covariance sequence $\{\Sigma_i\}$ satisfies scaling $O(h^{2p})$ for methods of strong order $p$ (Marumo et al., 2023).

• Gaussian Process Surrogates (GP):

In mesh convergence index (GCI) and Bayesian GP frameworks, the posterior variance at $h \to 0$ quantifies the uncertainty in the finest-grid prediction, e.g.

$$\mathrm{Var}\bigl(e(0)\mid \text{data}\bigr) = \sigma^2_\varepsilon(0)$$

where $\varepsilon$ is GP-modeled error and $\sigma^2_\varepsilon(0)$ is the posterior variance at zero mesh size (Bect et al., 2021).

• Multilevel Monte Carlo (MLMC):

MLMC telescopes the expectation across hierarchies of discretizations, redistributing computational cost to coarser levels while tightly controlling the sampling variance. The total variance is a weighted sum of variances at each level, and an optimal allocation minimizes cost for prescribed mean-square error (Vidal-Codina et al., 2014, Heng et al., 2021).

4. Error Variance Analysis in High-dimensional and Neural Architectures

In high-dimensional operator learning, quantification and control of discretization error variance have emerged as critical for stability, accuracy, and model efficiency:

• Fourier Neural Operator Layerwise Error:

The cumulative (layer-by-layer) error variance in FNOs propagates as $O(N^{-2s})$ under standard smoothness and weight regularity assumptions. Nonlinearities with higher smoothness (e.g., $C^\infty$ activation) and periodic positional encoding maximize achievable regularity, thereby accelerating aliasing-error convergence. Empirical standard deviation of discretization errors aligns with theoretical rates and highlights model-dependent smoothing effects (Lanthaler et al., 3 May 2024).

• Electronic Structure Error Cancellation:

In quantum chemistry calculations, error variance on energy differences (reaction energies, binding energies) is reduced by near-constant cancellation factors, though overall error decay rate remains unchanged. The dispersion of cancellation factor $Q_N$ is nearly independent of mesh cutoff, indicating robust predictability of difference computations (Cancès et al., 2017).

5. Practical Implications, Adaptive Strategies, and Limitations

Real-world applications demand calibrated assessment of discretization error variance to avoid over- or under-fitting, optimize computational efficiency, and guarantee credible uncertainty quantification:

• Adaptive Grid and Subsampling Schedules:

Discretization error variance may serve as a termination criterion for adaptive grid refinement or particle allocation—e.g., doubling grid points only when validation error plateaus and error variance estimates permit further improvement. This reduces training cost without loss of accuracy (Lanthaler et al., 3 May 2024).

• Error-aware Parameter Estimation:

Failure to account for discretization error variance can give rise to severely biased inverse estimates and unreliable uncertainty measures. Approaches such as IRLS + isotonic regression and dual coordinate-ascent in block Wishart optimization recover both parameters and local reliability efficiently (Matsuda et al., 2019, Marumo et al., 2023).

• Limitations:

Deterministic error analyses may fail to deliver meaningful variance estimates in settings without clear probabilistic interpretation or when all errors are measured as norms. Conversely, purely Bayesian covariances (e.g., Green-prior FE posteriors) can lack load-dependence unless post-hoc modifications (eigenvalue-rescaling) are performed (Poot et al., 2023). GCI methods provide only heuristic uncertainty quantification, lacking formal credible coverage (Bect et al., 2021).

6. Comparative Summary and Implementation Formulae

Below is a table summarizing key formulations for discretization error variance across principal domains:

Method/Domain	Error Variance Scaling	Notable Formula
Euler–Maruyama SDE	$O(h^2)$ per step; $O(h)$ trajectory	$\mathrm{Var}(\Delta X^h) \le K h^2$ (Jourdain et al., 2017)
Fourier Neural Operator	$O(N^{-2s})$, $s$=input regularity	$\|u_t - v_t\|_{\ell^2(X(N))} \le C N^{-s}$ (Lanthaler et al., 3 May 2024)
ODE Wishart Estimation	$O(h^{2p})$ for order-$p$ methods	$$\Sigma_i \sim \operatorname{Cov}[e_i],~e_i = x(t_i)-x_i$$ (Marumo et al., 2023)
Kalman Filtering	$O(h^{2\alpha})$	$$\mathbb{E}\|Q_k\tilde{x}_k - \hat{x}_k\|^2 \le C h^{2\alpha}$$ (Aalto, 2014)
Bayesian GP	Data-driven, mesh-size dependent	$$\mathrm{Var}(e(0) \mid \text{data}) = \sigma^2_\varepsilon(0)$$ (Bect et al., 2021)
MLMC bias control	Level-wise, adaptive	$V = \sum_\ell \frac{V_\ell}{N_\ell}$ (Vidal-Codina et al., 2014)

Discretization error variance, whether approached via classical deterministic theory or contemporary probabilistic and Bayesian frameworks, is a central quantity in computational model validation, statistical inference for dynamical systems, and uncertainty quantification for scientific computation and machine learning architectures. Conversely, the omission or misestimation of this variance can result in overconfident predictions, suboptimal resource allocation, or unreliable inference—necessitating careful analysis, adaptive methodology, and, where possible, full probabilistic treatment.