Lagged Backward-Compatible PINN (LBC-PINN)

• The paper demonstrates that LBC-PINN attains nearly the accuracy of full backward-compatible PINN at a significantly lower computational cost.
• It segments the time domain and employs transfer learning, ensuring continuity and stability in solving complex time-dependent PDEs.
• Empirical results on Allen–Cahn, Cahn–Hilliard, and unsaturated soil consolidation validate its effectiveness with substantial error reduction and efficiency gains.

A Lagged Backward-Compatible Physics-Informed Neural Network (LBC-PINN) is an advanced training protocol for physics-informed neural networks (PINNs), designed to efficiently and accurately solve nonlinear and/or higher-order time-dependent partial differential equations (PDEs) over long time horizons and across multiple physical regimes. LBC-PINN addresses the limitations of standard PINN approaches related to accuracy degradation, stiffness, and spectral bias when modeling evolution equations with complex temporal and spatial dynamics. By partitioning the time domain into segments and introducing a lagged backward-compatibility loss enforced only over a fixed-size window of recent segments, LBC-PINN achieves nearly the accuracy of full backward-compatible PINNs (bc-PINNs)—which enforce continuity over all previous segments—at substantially reduced computational cost (Mattey et al., 2021, Li et al., 3 Feb 2026).

1. Theoretical Foundations and Motivation

Conventional PINNs embed differential constraints directly into a neural network loss, enabling the approximation of a PDE solution $u(x, t)$ from spatio-temporal inputs, subject to initial and boundary conditions. However, for strongly nonlinear or higher-order equations—such as the Allen–Cahn or Cahn–Hilliard equations—the standard PINN formulation suffers substantial accuracy loss unless extremely fine sampling is used, which escalates computational demands. The LBC-PINN paradigm extends bc-PINN by introducing:

• Segmentation of the time domain into $N$ segments, possibly in logarithmic time to address multi-scale phenomena or stiff systems.
• Backward-compatibility enforcement over a lag window of the $J$ most recent segments, rather than the entire history, thereby reducing the computational burden without sacrifice of continuity.
• Transfer learning between segments via parameter initialization (weight transfer and optional freezing of early layers), accelerating convergence (Mattey et al., 2021, Li et al., 3 Feb 2026).

This protocol exploits the locality of temporal dependencies in many PDEs, constraining solution drift only in the most influential temporal neighborhood, which empirical evidence suggests suffices for both accuracy and stability.

2. Mathematical Formulation and Segmentation

Let $\Omega$ denote the spatial domain and $[0, T]$ the temporal domain. The time interval is partitioned as

$\Delta T_n = [T_{n-1}, T_n],\quad n=1,\ldots,N.$

For stiff or multi-scale PDEs, a logarithmic segmentation scheme is frequently employed: $$\log_{10}T_n = \log_{10}T_0 + n\frac{\log_{10}T_{\max} - \log_{10}T_0}{N}$$ allowing uniform resolution in log-time and optimal coverage of both rapid-transient and slow-dissipation regimes (e.g., unsaturated soil consolidation with air and water phase interactions) (Li et al., 3 Feb 2026).

Within each segment, a PINN is trained to approximate the solution over $\Omega\times\Delta T_n$, with transfer initialization using parameters from $\Delta T_{n-1}$. This approach efficiently aligns network focus with the solution's evolving temporal scale.

3. Loss Function Construction

For segment $n$, the composite objective is

$$L^{(n)}(\theta) = \omega_{IC}L^{(n)}_{IC} + \omega_{BC}L^{(n)}_{BC} + \omega_{R}L^{(n)}_{R} + \omega_{S} L^{(n)}_{S}$$

where:

• $L_{IC}$: initial condition loss,
• $L_{BC}$: boundary condition loss, including periodic or Dirichlet/Neumann enforcement,
• $L_R$: physics-informed residual, based on the governing PDE (e.g., Allen–Cahn, Cahn–Hilliard, or coupled hydro-mechanical equations),
• $L_S$: lagged backward-compatibility loss.

Lagged backward-compatibility loss enforces inter-segment continuity: $$L_S = \frac{1}{N_S} \sum_{s=1}^{N_S} \sum_{j=\max(1,\,n-J)}^{n-1} \lambda_{n-j} \left| u_\theta(x_s, t_s) - u^{(j)}(x_s, t_s) \right|^2$$ with $N_S$ sample points $(x_s, t_s)$ in $\Omega \times [T_{n-J}, T_{n-1}]$ and penalty weights $\lambda_j$. For $J = n-1$, $L_S$ recovers the original bc-PINN loss; for $J \ll n$, LBC-PINN is obtained.

The use of a logarithmic residual variant for $L_R$,

$$L_R^{(\ln)} = \frac1{N_r}\sum_{r} \ln\left[1 + R^2(x_r^R, t_r^R)\right]$$

mitigates the dominance of large-magnitude residuals and improves accuracy on initial and boundary conditions (Mattey et al., 2021).

4. Training Algorithm and Transfer Learning

The LBC-PINN algorithm proceeds as follows:

1. For segment $n=1$ to $N$:
   • Collect collocation points for initial/boundary conditions, PDE residuals, and snapshot points for backward-compatibility over $[T_{n-J},T_{n-1}]$.
   • Initialize the network parameters using those from the previous segment ($\theta^{(n-1)}$).
   • Optionally freeze early layers for a few iterations to maintain low-level features (Li et al., 3 Feb 2026).
   • Train via Adam and/or L-BFGS optimizers until convergence, targeting total loss $L^{(n)}$.
   • Store network outputs on snapshot grids for subsequent enforcement of $L_S$.
2. After all segments, assemble the piecewise solution.

This segment-wise protocol (see pseudocode in (Mattey et al., 2021, Li et al., 3 Feb 2026)) accelerates convergence and facilitates stable long-term integration by leveraging prior knowledge.

5. Numerical Performance and Validation

LBC-PINN achieves substantial performance gains relative to standard PINN and comparable accuracy to bc-PINN, with much reduced computational cost in $L_S$ evaluation:

• Allen–Cahn equation ($\Omega=[-1,1]$, $T=1$, 5 segments):
  • Relative $L^2$ error: PINN $0.992$, bc-PINN $0.070$, LBC-PINN ($J=2$) $0.085$.
  • Wall-clock time (V100 GPU): PINN 600 s, bc-PINN 420 s, LBC-PINN 300 s (Mattey et al., 2021).
• Cahn–Hilliard equation (20 segments):
  • Relative $L^2$ error: PINN $0.859$, bc-PINN (phase-space) $0.036$, LBC-PINN ($J=3$) $0.045$.
  • Wall-clock: PINN 800 s, bc-PINN 480 s, LBC-PINN 360 s (Mattey et al., 2021).

Unsaturated soil consolidation under long-term loadings ($T_{\max}=10^{10}$ s):

• With $N=5$ log-time segments, LBC-PINN achieves mean absolute error (MAE) $<5.6\times 10^{-3}$ kPa and $R^2>0.9995$ when compared to FEM benchmarks. Reducing segment number degrades accuracy sharply (MAE $0.23$ for $N=2$), illustrating the necessity of tailored segmentation (Li et al., 3 Feb 2026).

A simplified segmentation using the characteristic air-phase dissipation time $t_s = H^2/c_{v,a}$ decreases segment number and training cost by $20$–$30$\% with only moderate (~$10$\%) accuracy loss.

6. Architecture and Implementation Guidelines

Network architecture is typically a fully-connected feed-forward design:

• Inputs: $(x,t)$ or normalized $(z,t)$.
• Outputs: one or more physical fields (e.g., $u$, $u_a$, $u_w$).
• Hidden layers: $4 \times 200$ (general PDE), $5 \times 50$ (unsaturated soil).
• Activations: $\tanh$.
• Parameter initialization: Xavier.
• Optimizer: Adam (initial), then L-BFGS; loss tolerance $\sim 10^{-6}$.

Practical recommendations:

• Segmentation: select segment width so solution gradients per segment remain tractable (e.g., 4–5 segments for Allen–Cahn, 10–20 for Cahn–Hilliard).
• Sampling: $5000$–$10000$ collocation points per segment (CH), $15000$–$25000$ (AC); $2000$ each for initial/boundary conditions; $5000$ for backward compatibility (Mattey et al., 2021, Li et al., 3 Feb 2026).
• Lag parameter: $J=2$–$4$ is effective, balancing the backward-compatibility window and computational load.
• Penalty weights: can be uniform ($\lambda_k=1$) or exponentially decaying ($\lambda_k = \exp(-\alpha k)$).
• For higher-order PDEs, a phase-space split (auxiliary variables for derivatives) reduces autodifferentiation stack depth and cuts training effort by $30$–$50\%$ (Mattey et al., 2021).
• For multi-decadal or stiff systems, log-time segmentation and transfer learning are crucial (Li et al., 3 Feb 2026).

7. Applications and Sensitivity

LBC-PINN has been applied to:

• Nonlinear and higher-order reaction-diffusion equations (Allen–Cahn, Cahn–Hilliard) modeling phase separation and reactive transport.
• Hydro-mechanical coupled problems in geomechanics (unsaturated soil consolidation with strongly coupled air–water diffusion and nonlinear boundary conditions) (Li et al., 3 Feb 2026).

Robustness is maintained across parameter ranges. For unsaturated soil consolidation, LBC-PINN's MAE remains below $1.7 \times 10^{-2}$ and MRE below $0.18$ over air-to-water permeability ratios $k_a/k_w$ spanning $10^{-3}$ to $10^{3}$. For $k_a/k_w \geq 10$, errors drop below $4 \times 10^{-3}$ (MAE) and $0.05$ (MRE) (Li et al., 3 Feb 2026).

A plausible implication is that LBC-PINN is effective across a wide variety of stiff, multi-scale, and nonlocal PDE systems requiring high-accuracy, long-horizon simulation and inversion.

---

References:

• "A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations" (Mattey et al., 2021)
• "Lagged backward-compatible physics-informed neural networks for unsaturated soil consolidation analysis" (Li et al., 3 Feb 2026)