PeRCNN: Physics-Encoded Recurrent CNNs

• PeRCNN is a framework that integrates explicit PDE-based physics with recurrent CNN architectures to model spatiotemporal dynamics.
• It decomposes evolution into differentiation using fixed convolutional kernels and integration via numerical time-stepping for accurate state prediction.
• Experimental benchmarks show PeRCNN architectures achieve higher forecasting SNR, reduced RMSE, and superior interpretability compared to black-box models.

Physics-encoded Recurrent Convolutional Neural Networks (PeRCNNs) are a class of neural architectures that merge explicit inductive biases from spatiotemporal physics—primarily partial differential equations (PDEs)—with the expressive, data-driven modeling capacity of deep learning. These frameworks couple classical operator structure, such as differentiation and integration steps, with convolutional architectures for spatial locality and recurrence for temporal evolution. PeRCNNs have been shown to provide interpretable, stable, and generalizable modeling of high-dimensional dynamical systems, particularly when governing physics is only partially known or the system is strongly driven by unobservable, time-varying sources.

1. Core Architectural Principles

PeRCNN frameworks are unified by a cell architecture that explicitly encodes both data-driven and physics-based dynamics at each temporal step. Central to these designs is the decomposition of evolution into separate differentiator and integrator modules, each leveraging spatial convolutions and temporal recurrence. The primary steps are as follows:

• Differentiation (PDE right-hand side): Fixed or learnable convolutional kernels compute spatial derivatives (e.g., gradients $\nabla$, Laplacians $\Delta$), which are then combined with the current system state and, when available, additional physical descriptors (e.g., velocity fields or morphometry) in a CNN to approximate the PDE's right-hand side.
• Integration (time-stepping): Numerical integration (e.g., forward Euler, Heun’s, or Runge–Kutta methods) advances the state over a temporal increment using the output of the differentiator. Integration is augmented with a learnable correction module—often a small, residual convolutional network—to directly model higher-order errors and nonlinear discrepancies.
• Recurrent structure: The system state at each time step is passed through this differentiator–integrator sequence, forming a recurrent loop akin to classical time-marching PDE solvers.
• Memory augmentation: Some variants (e.g., PhICNet) maintain explicit buffers (e.g., “PDE-memory” $C_t$ for storing recent field observations, “source-memory” $C_{V,t}$ for history of estimated sources) to accommodate higher-order or non-Markovian dynamics (Saha et al., 2020).
• Physics encoding: Fixed finite-difference operators impose correct local structure for derivatives (e.g., using 4th-order central Laplacian stencils (Ren et al., 2021)), while boundary and initial conditions can be directly hard-encoded using boundary padding or ghost-node schemes.

2. Mathematical Formulation and Physics Incorporation

PeRCNNs are applied to systems described by inhomogeneous PDEs of the form:

$$\frac{\partial^n u}{\partial t^n} = F\left(\mathbf{x}, u, \nabla u, \nabla^2 u, \dots; \theta\right) + v(\mathbf{x}, t)$$

where $u(\mathbf{x}, t)$ is the physical field; $F$ encodes the known, possibly parameterized, homogeneous physics; and $v(\mathbf{x}, t)$ is an unknown source or perturbation. Typical PDEs include diffusion, wave, reaction–diffusion, Burgers’, and Navier–Stokes equations.

Explicit physics encoding is achieved through:

• Implementation of spatial derivatives using non-trainable, fixed convolutional filters (ensuring well-posed stencils).
• Partitioning of the field evolution into a “physics-explained” component (using $F$ and known parameters $\theta$, learned if necessary) and a residual component estimated by data-driven networks (e.g., $g(\cdot)$ or separate CNN blocks).
• Source identification by interpreting the residual between observed and physics-predicted fields as the spatiotemporally varying source (Saha et al., 2020).
• In systems with additional structure (e.g., velocity couplings), feeding both the state and auxiliary fields through respective differentiator–integrator towers (as in PARCv2 (Nguyen et al., 2024)).

Boundary and initial conditions are usually hard-encoded. Periodic boundaries use circular padding; Dirichlet and Neumann conditions use fixed value or ghost-cell padding at every convolution layer (Ren et al., 2021).

3. Loss Functions and Training Methodologies

Loss functions in PeRCNNs reflect their dual focus on trajectory fidelity and coerced physics-inductive behavior:

• State prediction loss: Enforces the match between predicted and true fields, typically via $L_2$ or $L_1$ norms.
• Source prediction loss: Penalizes discrepancies in the estimated source fields, when those are available for supervised evaluation (Saha et al., 2020).
• Physics-informed loss: Enforces minimal residual of the discretized PDE by evaluating the difference between numerical time derivatives and the physics operator (e.g., $$\mathcal{L}_{PDE} = \sum_{n,i,j} \left\|R_{n,i,j}\right\|^2$$, with $R_{n,i,j}$ the PDE residual at a gridpoint) (Ren et al., 2021).
• Sparsity penalty: When source fields are expected to be sparse, an $L_1$ norm may be added to favor parsimonious source identification (Saha et al., 2020).
• Two-stage training: Recent PeRCNNs, notably PARCv2, use sequential optimization: first train the differentiator (freezing the integrator) to predict instantaneous derivatives, then train the integrator (freezing the differentiator) for accurate long-range rollout (Nguyen et al., 2024).

Training is conducted using end-to-end backpropagation through time, typically with Adam or SGD, on synthetic or DNS-generated trajectory datasets. Batch size is usually limited by domain size (e.g., single full-grid samples). Learning rates and batch sizes are adapted to fit hardware constraints and problem difficulty (Saha et al., 2020, Ren et al., 2021, Nguyen et al., 2024).

4. Experimental Benchmarks and Quantitative Performance

PeRCNN architectures have been systematically validated on a variety of canonical and application-focused testbeds:

• Canonical PDEs: Problems include 2D heat diffusion, wave propagation, viscous Burgers’, and coupled reaction–diffusion systems. All employ synthetic ground truth generated via finite-difference solvers with known (Dirichlet, periodic, or Neumann) boundaries (Saha et al., 2020, Ren et al., 2021, Nguyen et al., 2024).
• Performance metrics: Common figures of merit are
  • Forecasting SNR: For example, PhICNet achieves up to 10 dB higher SNR at long prediction horizons compared to ConvLSTM or PDE-RNN+CNN baselines (Saha et al., 2020).
  • Root-mean-squared error (RMSE): In PARCv2, RMSE on Burgers’ flow is 0.0129 cm/s versus FNO’s 0.0289 and PhyCRNet’s 0.0588 (Nguyen et al., 2024).
  • Source correlation: Pearson correlation for estimated vs. true source remains above 0.9 in PhICNet, an order of magnitude above black-box baselines (Saha et al., 2020).
• Generalization and extrapolation: PeRCNNs exhibit robust accuracy outside the training window, both in time (long-range rollouts with minimal error growth (Ren et al., 2021)) and in system parameters (e.g., Reynolds/diffusivity variation in PARCv2 without catastrophic performance drop (Nguyen et al., 2024)).
• Material-physics tasks: PARC demonstrates three orders of magnitude speedup over DNS while maintaining quantitative agreement on sensitivity metrics (e.g., hotspot growth rates/QoIs in mesoscale energetic materials) (Nguyen et al., 2022).

5. Interpretability, Adaptivity, and Advantages

A defining feature of PeRCNNs is interpretability resulting from the explicit separation of physics-explained and data-driven dynamics:

• Physical parameter recovery: PhICNet and related models learn hidden PDE parameters (diffusivity $\alpha$, wave speed $c$, viscosity $\beta$) end-to-end, matching ground truth within a few percent (Saha et al., 2020).
• Source identification: The explicit source map $V_t$ in PhICNet allows recovery and analysis of unobservable forcing, with high spatial correlation to ground truth (Saha et al., 2020).
• Microstructure attribution: In PARC, pixel-level saliency for the morphology encoder can reproduce critical void statistics, providing physical insight into which microstructure features drive system response (Nguyen et al., 2022).
• Online adaptation: PhICNet can re-tune only the physical parameter vector $\theta$ post-training, accommodating slowly varying dynamics in the physical system without learning new source dynamics (Saha et al., 2020).

Key advantages over baseline architectures include stable long-term forecasting, superior interpretability, direct enforcement of boundary/initial constraints, and data efficiency.

6. Limitations and Open Research Directions

Despite their strengths, PeRCNNs exhibit several limitations and developmental frontiers:

• Requirement for explicit PDE structure: The form and temporal order $n$ of the governing PDE, as well as correct boundary condition implementations, must be specified a priori (Saha et al., 2020).
• Discretization constraints: Fixed finite-difference convolutions presuppose uniform spatial grids; extension to unstructured meshes or irregular samplings would necessitate graph convolutional or mesh-based operators (Saha et al., 2020, Nguyen et al., 2024).
• Global physical constraints: Unlike some PINN approaches, PeRCNNs do not inherently enforce certain global physical laws (e.g., incompressibility) unless extended with divergence-free projections or additional loss terms (Nguyen et al., 2024).
• Hyperparameter sensitivity: Cross-validation is required to tune hyperparameters such as source memory order $K$, sparsity weights, integrator correction depth, and skip-encoder intervals (Saha et al., 2020, Ren et al., 2021).
• Scope of source dynamics: Generalization to highly complex or high-dimensional source behavior, especially in presence of unobserved channels, remains an open avenue (Saha et al., 2020).

Suggested future research directions include hybridizing PeRCNNs with physics-informed losses in integrator stages, integrating equivariant convolutional modules for symmetry-rich problems, and applying adaptive time-stepping for stiff or multiscale systems (Nguyen et al., 2024).

7. Comparison of Representative Frameworks

Model	Physics Encoding Approach	Temporal Cell	Loss/Training Focus
PhICNet	PDE finite-diff + residual CNN	ConvRNN w/ source memory	$\mathcal{L}_{\rm pred}$, $\mathcal{L}_{\rm src\_pred}$, $\mathcal{L}_{\rm src\_sparse}$, online tuning (Saha et al., 2020)
PhyCRNet	Discretized PDE residual	Encoder–ConvLSTM–Decoder	Physics-informed loss ($\mathcal{L}_{PDE}$), hard BC/IC encoding (Ren et al., 2021)
PARCv2	Finite-diff stencils + hybrid integrator	Differentiator + Integrator (CNNs), no gating	2-stage: differentiator/integrator training, rollouts, $L_1$ (Nguyen et al., 2024)
PARC	PDE-motivated differentiator/integrator + morphology U-Net	Recurrent CNN (dX/dt + integrate)	State/derivative consistency, no physics penalty (Nguyen et al., 2022)

These frameworks illustrate the spectrum of PeRCNN realizations, unified by the explicit separation of physical priors and data-driven correction, leading to interpretability and robust, extrapolative performance for complex spatiotemporal dynamical systems.