Operator Neural ODEs (NODE-ONet) Overview

• Operator Neural ODEs are models that fuse neural operator architectures with traditional neural ODEs to model continuous-time dynamical systems.
• They employ innovative training schemes like derivative supervision and encoder–ODE–decoder frameworks to enhance accuracy and generalization in both ODE and PDE applications.
• Empirical results demonstrate reduced interpolation and extrapolation errors on benchmarks, highlighting improved sample efficiency and robustness in physics-informed simulations.

Operator Neural ODEs (NODE-ONet) generalize neural ordinary differential equations (NODEs) by incorporating operator learning and neural operator architectures into the modeling and training of continuous-time dynamical systems. NODE-ONet frameworks encompass approaches where the ODE vector field, the model training scheme, or the continuous depth architecture is enriched or regularized by neural operators, with applications that range from data-driven trajectory learning to physics-informed surrogate modeling for partial differential equations (PDEs) (Gong et al., 2021, Li et al., 17 Oct 2025, Cho et al., 2023).

1. Mathematical Formulation and Operator Perspective

The foundational element of NODE-ONet is the observation that the right-hand side of a neural ODE,

$$\frac{d\mathbf{h}(t)}{dt} = f_\theta(\mathbf{h}(t), t),$$

is, in fact, a differential operator acting on trajectories in a suitable function space. Operator-based NODEs recast $f_\theta$ as a neural operator

$$\mathcal{D}_\theta : \{\mathbf{h}(\cdot)\} \longrightarrow \{\xi(\cdot)\}, \qquad \frac{d\mathbf{h}(t)}{dt} = \mathcal{D}_\theta[\mathbf{h}(\cdot), t](t).$$

For time-dependent PDEs,

$$\partial_t u(t,x) + \mathcal{L}[a](u)(t,x) = f(t,x),$$

NODE-ONet operator learning seeks to approximate the solution map

$\Psi^\dagger: \{a, f, u_0, u_b\} \mapsto u$

via a composition of learned encoder, neural ODE evolution, and decoder (Li et al., 17 Oct 2025).

Table 1: Operator Neural ODEs—Formulation Comparison

Framework	Operator View	ODE Field Architecture
NDO-NODE (Gong et al., 2021)	NDO estimates $\dot{x}$	ODE vector field regularized
NODE-ONet (Li et al., 17 Oct 2025)	Enc-dec NODE for PDE operators	Latent ODE with physics-informed NN
BFNO-NODE (Cho et al., 2023)	Global integral operator	ODE RHS via Fourier Neural Op.

2. Neural Operator Architectures within NODE-ONet

Neural Differential Operator (NDO) Integration

In the NDO-NODE formulation, the standard NODE architecture is augmented with a pre-trained neural operator $\mathcal{D}_\phi$ for explicit derivative supervision. The loss function combines trajectory fit and derivative fit:

$$\mathcal{L}(\theta) = \sum_i \|x'(t_i; \theta) - x(t_i)\|^2 + \lambda \sum_i \|f_\theta(x(t_i), t_i) - \mathcal{D}_\phi(\{x(t_i)\}, \{t_i\})\|^2.$$

$\mathcal{D}_\phi$ is pre-trained on a function library composed of trigonometric polynomials, enabling it to serve as a robust surrogate for local derivatives (Gong et al., 2021).

Encoder–NODE–Decoder Frameworks for PDEs

NODE-ONet for operator learning in PDEs centers on an encoder–ODE–decoder paradigm:

• Encoder: Maps function-valued PDE parameters (e.g., $a$, $f$, $u_0$) onto low-dimensional latent representations $\bm v(t)$.
• Neural ODE: Evolves latent state $\bm\psi(t)$ with physics-encoded terms reflecting PDE structure (e.g., explicit bilinear, nonlinear, or polynomial couplings).
• Decoder: Lifts latent trajectories back to function space, either with fixed bases (Fourier, FEM) or neural networks (Li et al., 17 Oct 2025).

Fourier Neural Operator NODE (BFNO-NODE)

Branched Fourier Neural Operator (BFNO) layers are stacked in the neural ODE's right-hand side, replacing standard MLPs/CNNs. Each BFNO layer implements a frequency-domain convolution:

• Forward: $\widehat g = \mathcal{F}(g)$.
• Branched filters/aggregation: $O_i = R_i \odot \widehat g$, aggregated by a small FC net.
• Inverse: $\mathcal{C} = \mathcal{F}^{-1}(\rho(\widehat g))$.
• Skip and nonlinear activation: $g_{k+1} = \sigma(\mathcal{C} + Wg_k)$. This enables the ODE to express global, nonlocal operator dynamics (Cho et al., 2023).

3. Training Methodologies and Loss Functions

NDO-regularized NODE Training

• Pre-training: The NDO is optimized on synthetic trajectories with known derivatives.
• Supervisory regularization: Derivative estimates from the NDO inform NODE training via an auxiliary loss.
• Combined optimization: The NODE is trained with both trajectory and operator-derived losses, fixing NDO weights after pre-training (Gong et al., 2021).

Encoder–ODE–Decoder for PDE Solution Operators

• Loss: Discrete $L^2$ loss over mini-batches of spatiotemporal solution grids plus optional regularization.
• Temporal integration: Euler or Runge-Kutta methods applied to latent dynamics.
• Optimization: ADAM (learning rate $\sim 10^{-3}$) up to $10^5$ epochs; L-BFGS for refinement.
• Sensors: Inputs encoded via basis projections or sensor sampling (Li et al., 17 Oct 2025).

Operator NODEs with Global Convolution

• ODE solver: Supports explicit or adaptive integration routines (e.g., Dormand–Prince).
• Loss functions: Classification (cross-entropy), normalizing flow (NLL), or time-series (accuracy/AUROC).
• Regularization: Standard weight decay; BFNO-NODE demonstrated stable training, low NFE, and rapid convergence (Cho et al., 2023).

4. Theoretical Guarantees and Error Analysis

Theoretical results include:

• Stability: Derivative-regularized NODEs reduce sensitivity to ODE solver tolerance, particularly improving training robustness in stiff or chaotic systems by anchoring the learned vector field to operator-informed derivative estimates (Gong et al., 2021).
• Approximation bounds: For NDO-NODE, the $L^1$ error between the operator’s derivative estimates and ground-truth derivatives admits explicit bounds in terms of the network Lipschitz constant, basis approximation, and training error (Gong et al., 2021).
• Operator-learning enc/dec error: In encoder–NODE–decoder architectures, the total solution error is bounded by the sum of encoding/decoding errors, the Hölder continuity of the map, and neural approximation error. For mesh-based encoders, the error decays as $O(h^\alpha)$ (Li et al., 17 Oct 2025).

5. Empirical Results and Applications

NODE-ONet variants exhibit improved sample efficiency, accuracy, and generalization across multiple regimes.

ODE Benchmarks

• Planar spiral system: NDO-NODE reduces interpolation MSE from $0.026 \to 0.012$ and extrapolation MSE from $4.52 \to 0.52$.
• Damped oscillator and three-body problem: NDO-NODE consistently outperforms standard NODEs in both interpolation and extrapolation test error (Gong et al., 2021).
• Stiff ODEs: NDO-NODE accurately captures rapid transitions, outperforming both vanilla NODE and other regularized NODEs (Gong et al., 2021).
• F-16 aircraft vibration: RMSE reduced from $\sim 0.25$ to $\sim 0.18$ (first-order) and from $\sim 0.20$ to $\sim 0.14$ (second-order) (Gong et al., 2021).

PDE Operator Learning

• 1D nonlinear diffusion–reaction: Relative error as low as $2.7 \times 10^{-3}$ for source-to-solution, outperforming DeepONet and MIONet with fewer parameters and coarser time grids.
• 2D Navier–Stokes: Maintains low error for both interpolation and significant temporal extrapolation, outperforming prior operator networks (Li et al., 17 Oct 2025).
• Transferability: Decoder modules trained on one PDE variant can often be reused for related equations; extrapolation to longer time frames is stable.

Operator NODEs in Deep Learning Tasks

• Image classification: BFNO-NODE achieves CIFAR-10 test accuracy $62.9\%$ (vs $62.6\%$ best other NODE), with fewer function evaluations per integration.
• Time-series and normalizing flows: Consistent improvements in classification accuracy and bits/dim over NODE and RNODE baselines (Cho et al., 2023).

6. Architectural and Practical Insights

• Physics encoding: Embedding physically meaningful structure directly in the NODE parameterization reduces parameter count, improves extrapolation, and injects strong inductive bias for system identification (Li et al., 17 Oct 2025).
• Global operator expressivity: BFNO and related neural operator layers provide a functional space generalization of conventional NODE right-hand sides, enabling global convolutional effects and improved integration properties (Cho et al., 2023).
• Parameter efficiency and robustness: NODE-ONet frameworks are consistently more parameter-efficient, requiring fewer training samples and resources to attain superior generalization, especially for operator regression and out-of-distribution prediction.

7. Limitations and Extensions

• Domain constraints: BFNO-NODE and FNO-based methods require regular domain grids for FFT operation. Extensions to irregular domains demand further development (e.g., graph neural operators) (Cho et al., 2023).
• NDO coverage: The accuracy of NDO-regularized NODEs depends on the expressiveness of the pre-training function library; poor basis selection can limit derivative supervision fidelity (Gong et al., 2021).
• Computational complexity: FFT layers in BFNO incur $O(d_g \log d_g)$ complexity, which may be challenging for very high-dimensional states (Cho et al., 2023).
• Operator transferability: While decoder modules are reusable across related problems, their universality for highly nonlinear or discontinuous PDE families remains to be established (Li et al., 17 Oct 2025).

NODE-ONet approaches constitute a unified direction in continuous-depth learning, integrating operator-based supervision, efficient encoder–latent–decoder paradigms, and spectral operator architectures for robust, parameter-efficient modeling of both ODE and PDE dynamics (Gong et al., 2021, Li et al., 17 Oct 2025, Cho et al., 2023).