PhICNet: Physics-Incorporated Conv RNN

• PhICNet is a hybrid framework integrating physics-based PDE models with deep learning to forecast spatio-temporal dynamics and identify hidden sources.
• It employs a convolutional RNN core combined with a RED-Net for residual source prediction, achieving remarkable SNR improvements and high spatial correlation in source recovery.
• The method is validated on canonical systems like heat diffusion, wave equations, and 2D Burgers’ flow, demonstrating robust forecasting performance and adaptability to noise and parameter shifts.

PhICNet (Physics-Incorporated Convolutional Recurrent Neural Network) is a hybrid modeling framework combining numerical physics-based models and deep learning for the joint tasks of spatio-temporal forecasting and source identification in dynamical systems governed by partial differential equations (PDEs) with unobservable, time-varying external sources. PhICNet formulates the underlying PDEs as convolutional RNN cells in a neural architecture, enabling data-driven learning of both the system evolution and hidden source terms. The framework is end-to-end trainable and yields state-of-the-art performance in forecasting and source recovery across a range of simulated physical systems (Saha et al., 2020).

1. Mathematical Formulation

PhICNet models a spatio-temporal field $u(x,y,t)$ on a domain $\Omega \subset \mathbb{R}^2$ governed by an inhomogeneous PDE of order $n$ in time: $$\frac{\partial^n u}{\partial t^n} = F(x,y,u,\partial_x u, \partial_y u, \partial_{xx} u, \ldots; \theta) + v(x,y,t), \tag{1}$$ where $F$ is a known function (potentially nonlinear) of the field and its spatial derivatives, parameterized by unknown physical coefficients $\theta$ (such as diffusivity $\alpha$, wave speed $c$, or viscosity $\beta$). The unobservable source $v(x,y,t)$ evolves via a separate $k$th-order equation: $$\frac{\partial^k v}{\partial t^k} = G(x,y,v). \tag{2}$$ Spatial discretization uses a regular $X \times Y$ grid with $U_t \in \mathbb{R}^{X\times Y}$ and finite-difference approximations for temporal and spatial derivatives. Small (e.g., $3 \times 3$) fixed convolutional kernels $D_{jl}$ implement spatial differencing, such as

$$D_{20} = \begin{pmatrix} 0 & 0 & 0 \ 1 & -2 & 1 \ 0 & 0 & 0 \end{pmatrix}, \;\; D_{02} = \begin{pmatrix} 0 & 1 & 0 \ 0 & -2 & 0 \ 0 & 1 & 0 \end{pmatrix}. \tag{3}$$

Temporal evolution is encoded in a convolutional RNN cell: the cell state $C_t \in \mathbb{R}^{n\times X\times Y}$ stacks $n$ previous maps. The cell computes a homogeneous PDE prediction $H_t$ by aggregating finite-difference temporal coefficients $w_{hc}^p$ and a (learnable) sum of weighted convolutional terms from the PDE's right-hand side: $$H_t = \sum_{p=1}^n w_{hc}^p C_t^p + f(D_{10}*U_t, D_{01}*U_t, D_{20}*U_t, \ldots; \theta). \tag{6}$$ The source term is estimated internally as a residual $V_t = U_t - H_{t-1}$ and buffered in $C_{V,t} \in \mathbb{R}^{k \times X \times Y}$ for prediction of future sources via a RED-Net (residual encoder-decoder) network, yielding

$$\widehat V_{t+1} = \sum_{p=1}^k w_{vc}^p C_{V,t}^p + g(C_{V,t}), \tag{9}$$

with $g(\cdot)$ parameterizing nonlinear source dynamics $G$. The final prediction is the sum $\widehat U_{t+1} = H_t + \widehat V_{t+1}$.

2. Network Architecture

PhICNet comprises two principal components:

• PDE-RNN Core: Receives input $U_t$, maintains cell state buffer $C_t$ for the last $n$ frames, and source buffer $C_{V,t}$ for the last $k$ frames of estimated sources. All spatial operations leverage depth-wise 2D convolutions with fixed $3\times3$ differential kernels. Physical parameters $\theta$ are trainable scalars.
• RED-Net for Source Dynamics: Adopts a classical residual encoder–decoder topology with symmetric skip connections. The encoder includes $M$ down-sampling blocks with two convolutional layers (typical dimensionality 64) and ReLU, mirrored by up-sampling blocks in the decoder. Output is added to the linear finite-difference source forecast.

The RED-Net refines the nonlinear evolution of the source term $v$ and allows end-to-end learning of complex time-varying perturbations.

3. Training and Loss Functions

Training sequences $\{U_0, ..., U_T\}$ are used to forecast $\{\widehat U_{n+k}, ..., \widehat U_T\}$ and estimate sources $\{\widehat V_{n+k}, ..., \widehat V_T\}$. The composite loss function integrates:

• Prediction Error:

$$\mathcal{L}_{\mathrm{pred}} = \frac{1}{T-(n+k)+1} \sum_{t=n+k}^T \|U_t - \widehat U_t\|_2^2. \tag{11}$$

• Source-Prediction Consistency:

$$\mathcal{L}_{\mathrm{src\,pred}} = \frac{1}{T-(n+k)+1} \sum_{t=n+k}^T \|V_t - \widehat V_t\|_2^2, \quad V_t = U_t - H_{t-1}. \tag{12}$$

• Source Sparsity (optional):

$$\mathcal{L}_{\mathrm{src\,sparse}} = \frac{1}{T-(n+k)+1} \sum_{t=n+k}^T \|\widehat V_t\|_1. \tag{13}$$

Total loss: $$\mathcal{L} = \mathcal{L}_{\mathrm{pred}} + \mathcal{L}_{\mathrm{src\,pred}} + \lambda\,\mathcal{L}_{\mathrm{src\,sparse}}. \tag{14}$$ Optimization employs BPTT over all frames, with SGD or Adam at learning rate $10^{-3}$, batch size 4–8, and weight decay $10^{-6}$. Early stopping uses the validation loss.

4. Source Identification and Mechanisms

At each time step, PhICNet computes the homogeneous prediction $H_{t-1}$, calculates the empirical source residual $V_t = U_t - H_{t-1}$, updates the source buffer, and predicts the next-step source $\widehat V_{t+1}$ via RED-Net and finite-difference aggregation. Training the source consistency loss encourages $\widehat V_t \approx V_t$, and imposing $\mathcal{L}_{\mathrm{src\,sparse}}$ enforces known sparsity. This mechanism enables direct recovery of the unobservable external sources driving the physical system, with internal source estimates accessible as by-products of forecasting.

5. Experimental Evaluation

PhICNet is evaluated on three canonical PDE systems, each discretized on $64 \times 64$ grids across a time horizon of $T=200$ steps, with $20\%$ validation hold-out.

• Heat Diffusion ($n=1$):

$v$ is block-sparse and diffusive; 100 training, 50 test sequences.

• Wave Equation ($n=2$):

$v$ comprises two point oscillators at random locations; 300 training, 50 test sequences.

• 2D Burgers’ Flow ($n=1$, vector):

$v$ updates two Gaussian pressure zones traveling in circles; 300 training, 50 test sequences.

Boundary conditions: Dirichlet (heat, wave) and Neumann (Burgers).

Performance is measured by:

• Forecast Accuracy: SNR in dB,

$$\mathrm{SNR}(U_t, \widehat U_t) = 20 \log_{10} \left( \frac{\|U_t\|_2}{\|U_t-\widehat U_t\|_2} \right). \tag{15}$$

• Source Identification: Spatial correlation $$\rho(V_t^{\mathrm{true}}, V_t^{\mathrm{est}}) \in [-1,1]$$.

PhICNet is compared against three baselines: PDE-RNN + CNN, ConvLSTM, and RED-Net. The table below summarizes SNR and source correlation at $t=150$ averaged over test sets (95% CI):

System	Metric	PDE-RNN+CNN	ConvLSTM	RED-Net	PhICNet
Heat	SNR@150	12 dB (±1)	18 dB (±2)	24 dB (±1.5)	30 dB (±1)
	Corr(V)@150	0.40 (±0.05)	—	—	0.85 (±0.03)
Wave	SNR@150	8 dB (±1)	14 dB (±1.8)	20 dB (±1.2)	26 dB (±1)
	Corr(V)@150	0.30 (±0.06)	—	—	0.80 (±0.04)
Burgers	SNR@150	10 dB (±1)	16 dB (±2)	22 dB (±1)	28 dB (±1)
	Corr(V)@150	0.35 (±0.05)	—	—	0.78 (±0.05)

PhICNet achieves superior forecast longevity (SNR increases of 6–10 dB above baselines) and is uniquely effective in recovering unobservable source terms with spatial correlation $\rho \approx 0.8$–$0.9$.

6. Analysis: Hyperparameters, Robustness, and Adaptation

• Sparsity Penalty $\lambda$:

Forecasting and source identification peak for $10^{-3} \leq \lambda \leq 10^{-1}$. Excessive $\lambda$ causes over-sparsification; insufficient values diminish identifiability.

• Source Order $k$:

Optimal $k$ matches the true source ODE order (e.g., $k=2$ for wave experiments), whereas mismatch ($k=1$ or $k=3$) degrades performance.

• Noise Robustness:

Adding Gaussian noise up to $10\%$ of data variance leads to SNR degradation of less than 2 dB.

• Online Adaptation of Physical Parameters:

Physical coefficients $\theta$ (e.g., $\alpha$, $c$) can be adapted online when discrepancies exceed a threshold, allowing model re-tuning while keeping RED-Net fixed. Empirical parameter trajectories closely follow ground-truth time-varying profiles.

7. Implementation Protocol

Spatial discretization is performed on $[0,1]^2$ to $64 \times 64$ grids ($\Delta x = \Delta y = 1/63$), with timestep $\Delta t$ subject to the CFL condition. Differential kernels $D_{jl}$ are statically assigned; physical parameters $\theta$ are randomly initialized. PyTorch is used for network implementation, buffers $C_t$ and $C_{V,t}$ are realized via FIFO queues. RED-Net depth is set to $M=4$ with 64 channels and ReLU activations. Training is performed for 100–200 epochs with early stopping on validation loss.

This configuration permits direct reproduction of the forecasting and hidden source identification results across arbitrarily parameterized dynamical systems governed by PDEs of the form $(F,G)$ (Saha et al., 2020).