---
title: Time-Integrated Deep Operator Networks
url: https://www.emergentmind.com/topics/time-integrated-deep-operator-networks-ti-deeponets
type: topic
---

# Time-Integrated Deep Operator Networks

Time-Integrated Deep Operator Networks (TI-DeepONets) are neural operator frameworks that achieve stable, accurate, and temporally causal surrogate modeling of dynamical systems and partial differential equations (PDEs). Distinct from conventional DeepONet paradigms—full rollout and autoregressive mapping—TI-DeepONet reformulates the operator-learning objective to approximate instantaneous time derivatives, which are then integrated via classical numerical schemes. This separation of neural approximation and integration imposes a physics-informed, Markovian structure that substantially mitigates error propagation and enables reliability on long temporal horizons, including extrapolation far beyond the training interval [2505.17341][2508.05190][2511.08992].

## 1. Operator Learning Paradigm and Motivation

Classic DeepONet architectures learn operators that map an input function (such as an initial or boundary condition) into a target function (such as a time-evolved solution) via two subnetworks: a branch network for function encoding and a trunk network for querying the target domain [2206.06536]. In temporal settings, state prediction is commonly framed in two ways:

- **Full rollout (FR)**: Learn $u(x, t_0) \mapsto u(x, t)$ for $t$ over a fixed interval. FR ignores temporal causality and is unable to generalize to $t$ outside the training domain.
- **Autoregressive (AR)**: Model $u^n \mapsto u^{n+1}$ sequentially. AR suffers from accumulated error across steps, leading to instability in long-term prediction.

TI-DeepONet circumvents these issues by shifting the learning target to the operator generating the instantaneous time derivative $u_t = \mathcal{F}(x, t, u, \nabla u, ...)$, so that time evolution proceeds via integration (Euler, Runge–Kutta, Adler–Bashforth/Moulton, etc.), tightly coupling neural operator learning with numerical analysis [2505.17341][2511.08992]. This enforces temporal causality and renders the approach suitable for continuous-time and long-horizon tasks.

## 2. Architectural Features of TI-DeepONets

The canonical TI-DeepONet implements the neural operator $\mathcal{G}_\theta$ to map the current solution field $u^n$ into the instantaneous derivative $\dot{u}^n$ via a DeepONet structure:

- **Branch network**: Ingests values $\{u^n(\eta_j)\}_{j=1}^m$ at sensor points, producing coefficients $b \in \mathbb{R}^p$.
- **Trunk network**: Receives a query $(x, t)$ or $x$, producing $t(x) \in \mathbb{R}^p$.
- **Projection**: The output is $\dot{u}^n(x) \approx \sum_{i=1}^p b_i t_i(x)$ [2505.17341][2508.05190][2511.08992].

Extended variants incorporate dual branches for control inputs [2511.08992], dual outputs to predict both current state and the tangent (time-derivative) field [2508.05190], and trunk networks adaptable to spatial or spatio-temporal contexts.

Physics-informed versions (PITI-DeepONet) introduce additional loss heads to enforce PDE residuals, initial/boundary conditions, and consistency between network-predicted and autodifferentiated time derivatives [2508.05190]. TI(L)-DeepONet extends the core by learning the integration scheme’s coefficients via an auxiliary MLP, dynamically adapting Runge–Kutta weights to local dynamics [2505.17341]. This imparts further robustness in stiff or highly nonlinear regimes.

## 3. Embedded Time Integration Methods

Following neural operator inference, TI-DeepONet advances the field via a classical time-stepping integrator:

- **Forward Euler**: $u^{n+1} = u^n + \Delta t\, \mathcal{G}_\theta(u^n)$.
- **RK4**:
  $$
  \begin{aligned}
    k_1 &= \mathcal{G}_\theta(u^n) \\
    k_2 &= \mathcal{G}_\theta(u^n + \frac{\Delta t}{2} k_1) \\
    k_3 &= \mathcal{G}_\theta(u^n + \frac{\Delta t}{2} k_2) \\
    k_4 &= \mathcal{G}_\theta(u^n + \Delta t\, k_3) \\
    u^{n+1} &= u^n + \frac{\Delta t}{6}(k_1 + 2k_2 + 2k_3 + k_4)
  \end{aligned}
  $$
- **Multi-step Adams–Bashforth/Moulton** (used in TI-DeepONet at inference for accuracy and efficiency tradeoff) [2505.17341][2508.05190].

TI(L)-DeepONet replaces fixed RK weights with data-driven combinations, with learnable weights $\tilde{\alpha}_i$ satisfying $\sum_i \tilde{\alpha}_i = 1$ determined via a softmax-parameterized MLP, yielding
$$
u^{n+1} = u^n + \Delta t \sum_{i=1}^4 \tilde{\alpha}_i k_i
$$
[2505.17341]. This adaptivity enables the integrator to mitigate neural approximation errors and respond to solution stiffness.

## 4. Training Protocols and Loss Functions

Training in TI-DeepONet frameworks involves sampling trajectories from numerical solvers and constructing dataset pairs $(u^n, \partial_t u^n)$. The loss is typically mean squared error between predicted and true derivatives or next states [2505.17341][2511.08992]. 

Physics-informed variants (PITI-DeepONet) augment the loss with:

- PDE residuals: Enforce $\hat{u}_t^n(x)$ to match the PDE right-hand side evaluated on $\hat{u}^n$.
- Initial/boundary condition penalties
- Consistency loss between $\hat{u}_t^n$ and $\partial_t \hat{u}^n$ via AD
- Reconstruction errors $\|\hat{u}^n - u^n\|^2$ [2508.05190].

When used in optimal control (e.g., Differentiable Predictive Control, DPC [2511.08992]), the operator is pretrained and kept frozen. Policy learning proceeds by differentiating control loss through the surrogate-integrated dynamical system, leveraging automatic differentiation for efficient gradient estimation.

## 5. Performance Benchmarks and Comparison with Classical Schemes

TI-DeepONet and its variants are systematically benchmarked against FR and AR DeepONet baselines across canonical systems (Burgers’, KdV, Allen–Cahn, Lorenz, predator–prey, cart–pole):

- **Error reduction**: TI-DeepONet achieves 81–98% reduction in mean relative $L_2$ extrapolation error over AR and FR approaches across various PDEs [2505.17341][2508.05190].
- **Stability horizon**: Accurate prediction is sustained for up to twice the training interval; AR and FR baselines diverge rapidly outside the training domain [2505.17341].
- **Robustness**: Adaptive weights in TI(L)-DeepONet further reduce errors, especially in stiff/chaotic regimes [2505.17341].
- **Control applications**: In differentiable predictive control, policies trained with TI-DeepONet surrogates achieve terminal tracking errors $\mathcal{O}(10^{-4})$ for parabolic and reaction-diffusion PDEs, and 77% reduction in Burgers’ curvature cost, outperforming FDM-based rollouts in computational efficiency [2511.08992].
- **Physics-informing**: Residual-based self-assessment (PITI-DeepONet) offers online error proxies highly correlated (Pearson $\rho \approx 0.997$) with true prediction errors [2508.05190].

### Representative $L_2$ Error Table (Extrapolation)

| Problem      | Method              | $T^*$ Final Rel. $L_2$ Error |
|--------------|---------------------|------------------------------|
| Burgers 1D   | TI(L)-DeepONet      | 0.0462                       |
|              | TI-DeepONet         | 0.0579                       |
|              | Full Rollout        | 0.3281                       |
|              | Autoregressive      | 1.7154                       |
| KdV 1D       | TI-DeepONet         | 0.1941                       |
| Burgers 2D   | TI-DeepONet         | 0.1736                       |

Data: [2505.17341]; $T^*$ denotes final extrapolation time.

## 6. Extensions, Generalizations, and Limitations

TI-DeepONet formulations include:

- Physics-informed extensions (PITI-DeepONet), enforcing tangent-space PDE structure and OOD state detection [2508.05190].
- Bayesian and LSTM-based local operator variants for ODE and irregular, real-time settings (B-LSTM-MIONet) with uncertainty quantification by replica-exchange SGLD [2311.16519].
- Integration within differentiable control pipelines (TI-DeepONet+DPC), enabling offline policy optimization for PDE-constrained problems [2511.08992].
- Learnable integrator coefficients (TI(L)-DeepONet) for state-adaptive time integration [2505.17341].

Limitations include potential error accumulation in extremely long rollouts without regularization, ongoing computational overhead for very high-dimensional systems, and the need for careful hyperparameter selection in network and integrator design [2206.06536][2505.17341]. TI-DeepONet currently does not directly address stiff or implicit time-integration but formulation extensions for such settings are identified as promising directions [2206.06536].

## 7. Theoretical Guarantees and Outlook

Under standard assumptions (Lipschitz continuity, input discretization error, neural universal approximation), TI-DeepONet admits formal error bounds on both local and cumulative prediction error, with stronger stability properties than AR or FR DeepONet [2206.06536]. The structure ensures that error growth is limited by the integrator stability properties, and embedding the numerical integrator into the learning loop further aligns gradient flow with long-term predictive goals [2505.17341][2508.05190].

Future directions include implicit/multistep integrator variants, Bayesian uncertainty quantification, hybrid physics/data training, and application to networked, high-dimensional, and multi-agent PDE systems [2206.06536][2311.16519][2508.05190][2505.17341][2511.08992]. Combining the TI-DeepONet paradigm with model-based reinforcement learning and offline policy synthesis continues to drive advances in operator-based control of complex dynamical systems [2511.08992].

Source: https://www.emergentmind.com/topics/time-integrated-deep-operator-networks-ti-deeponets