PIDL-CF: Physics-Informed Deep Learning

• PIDL-CF is a hybrid framework that integrates empirical data with physical laws by coupling neural network outputs to PDE and constitutive residuals.
• It employs diverse architectures such as feedforward MLPs, composite LSTM-FNN structures, and physics-preserved CNNs to model state fields and unknown constitutive functions.
• Robust optimization techniques, adaptive sampling, and automatic differentiation enable PIDL-CF to infer unobserved physics accurately even under data scarcity.

Physics-Informed Deep Learning (PIDL-CF) encompasses a suite of techniques in which deep neural networks (DNNs) are trained not only on empirical data but also under explicit physical constraints—typically in the form of governing partial differential equations (PDEs) or constitutive relationships. The “CF” shorthand is applied—Editor's term—when the focus is on learning constitutive functions or unknown physical laws within or alongside state evolution, as in parameter estimation, material modeling, and closure discovery. In PIDL-CF, all network outputs and latent parameterizations are coupled via a loss function that penalizes deviations from both measured data and the residuals of PDEs or the thermodynamic laws. The approach has demonstrated robustness to data scarcity, capacity to infer unobserved physics directly from state fields, and superior performance compared to conventional inversion or purely supervised deep learning, particularly in scientific and engineering contexts (Tartakovsky et al., 2018, Bahtiri et al., 2024).

1. Neural Architectures for Constitutive and State Representation

PIDL-CF leverages a variety of neural architectures depending on the targeted physical system and the nature of unknown quantities to be recovered:

• Feedforward MLPs (Multilayer Perceptrons): Standard in early PIDL-CF formulations for state fields $u(\mathbf x)$, spatially-dependent coefficients $K(\mathbf x)$, or scalar constitutive laws $K(u)$ (Tartakovsky et al., 2018). Typical architectures use 2 hidden layers, 50 neurons per layer, tanh activations, Xavier initialization, and L-BFGS-B optimization.
• Composite Structures: In viscoelastic-viscoplastic constitutive modeling, a recurrent LSTM encodes deformation history, feedforward nets map this history to internal variables, and a further FNN parameterizes the free energy function, facilitating objectivity, symmetry, and thermodynamic consistency. The free energy network employs softplus output activation to enforce nonnegativity (Bahtiri et al., 2024).
• Mixed-variable Schemes: For incompressible flows, architectures predicting a stream function and stress tensor (not velocity and pressure directly) allow PDE constraints to be implemented via first derivatives, improving trainability and accuracy in laminar flow simulation (Rao et al., 2020).
• Physics-Preserved Convolutional Architectures: In multi-resolution PDE-preserved neural networks, discrete approximations of differential operators (e.g., convolutional filters matching finite difference stencils for Laplacian or advection) are fixed and combined with a trainable CNN backbone. The output is the roll-forwarded field under both learned and exact physics (Liu et al., 2022).
• Physics-Informed Generative Models: Recent advances incorporate normalizing flows and diffusion models as the network backbone, with physics losses enforcing PDE consistency during generation for uncertainty quantification and temporal/spatio-temporal prediction (Mo et al., 2022, Qiu et al., 2024).

2. Construction of Physics-Informed Loss Functions

A hallmark of PIDL-CF is the fully coupled loss, which integrates measurement fidelity, boundary/initial condition enforcement, and minimization of PDE or constitutive residuals:

$$L(\theta, \gamma) = L_{\rm data}(\theta, \gamma) + L_{\rm BC}(\theta, \gamma) + L_{\rm PDE}(\theta, \gamma)$$

• Data Fit ($L_{\rm data}$): Penalizes mismatch on observed states or parameters.
• Boundary/Initial Conditions ($L_{\rm BC}$): Penalizes violation of Dirichlet/Neumann and initial data.
• PDE Residual ($L_{\rm PDE}$): Enforces satisfaction of the governing PDE at collocation points, with derivatives obtained via automatic differentiation.
• Thermodynamic/Dissipation Losses: In models involving complex materials, additional penalty terms such as ReLU(-dissipation) are included to guarantee nonnegative entropy production (Bahtiri et al., 2024).
• Sparsity and Regularization: In discovery settings, sparsity is promoted on differential operator coefficients via $\ell_0$ penalties and sequential thresholded regression (STRidge) (Chen et al., 2020).

Loss terms may be weighted, and tuning is essential for stability and accuracy, particularly in multi-physics or composite models.

3. Parameterization and Learning of Constitutive Functions

PIDL-CF brings a unified treatment to both parameter inference and full functional discovery:

• Unknown Coefficients: Space-dependent parameters (e.g., diffusion coefficient $K(\mathbf x)$) are parameterized as neural networks, constrained by available data and physics via the coupled loss (Tartakovsky et al., 2018).
• Constitutive Law Discovery: When the functional form $K(u)$ is entirely unknown, a neural network mapping scalar field values to $K(\mathbf x)$0 is trained, with no direct observations required; the physics constraints facilitate learning from state observations alone (Tartakovsky et al., 2018).
• History-Dependent Constitutive Models: For viscoelastic/viscoplastic composites, LSTM/FNN architectures capture path-dependent internal variables and enforce invariance/objectivity by encoding all free energy dependencies through invariants and internal variables (Bahtiri et al., 2024).
• Thermodynamic and Objectivity Constraints: The architecture directly encodes requirements such as positive-definite free energy and satisfaction of the Clausius–Duhem inequality via softplus activations and explicit penalties (Bahtiri et al., 2024).

4. Optimization, Sampling, and Robustness

The state of the art in PIDL-CF dictates careful choices for optimization and data handling:

• Optimizers: L-BFGS-B is standard for deterministic settings while Adam is used for stochastic or large-scale/minibatch training.
• Collocation Sampling: Uniform or Latin hypercube sampling is used to select collocation points for PDE residual enforcement. Adaptive schemes are beneficial in regions of high solution gradient or discontinuity.
• Measurement Noise: Robustness to measurement noise is achieved by the physics-based regularization; the PDE constraint itself serves as an implicit denoiser, and empirical results confirm consistent accuracy under moderate data corruption (Tartakovsky et al., 2018).
• Automatic Differentiation: All required spatial and temporal derivatives are computed as differentiable graph operations, removing the need for analytic Jacobians or adjoint equations.
• Strong vs. Weak Condition Imposition: Boundary and initial conditions can be enforced strongly by architectural design (e.g., output basis functions zero at the boundaries) or weakly as loss penalties, with both methods supported (Tyagi et al., 2024).

5. Numerical Performance, Comparison, and Applications

PIDL-CF exhibits systematic improvements over both conventional inversion methods and black-box neural models:

Application	PIDL-CF Error	Conventional Error	Reference
Linear Diffusion (K)	$K(\mathbf x)$1	$K(\mathbf x)$2 (MAP inversion)	(Tartakovsky et al., 2018)
Nonlinear Diffusion (K(u))	$K(\mathbf x)$3 (noiseless)	Robust to 1% noise	(Tartakovsky et al., 2018)
Viscoelastic Composite, 3D	<5 MPa$K(\mathbf x)$4 MSE	Matches experimental/classical	(Bahtiri et al., 2024)
Traffic Flow Estimation	2–5% lower $K(\mathbf x)$5 error	Local PINN baseline	(Huang et al., 2023)
Car-Following	Up to 16.8% lower MSE (scarce data)	Black-box NN	(Mo et al., 2020)

PIDL-CF achieves high fidelity in reconstruction and generalization, particularly when direct measurements of constitutive relationships are unavailable or data are limited. In thermodynamic modeling, it extrapolates across loading conditions, compositions, and environmental states, and in traffic modeling, it outperforms classical and black-box deep learning baselines even under sparse sensor data (Mo et al., 2020, Huang et al., 2023). In parameteric PDE families, learned solvers drastically accelerate convergence and enhance stability compared to conventional PINN optimizers (Boudec et al., 2024).

6. Extensions, Generalization, and Limitations

PIDL-CF methodology is widely extensible:

• History-Dependence and Multiphysics: Inclusion of recurrent architectures and comprehensive state variables permits modeling of path-dependent and multi-physics coupled systems (e.g., fiber/matrix composites, poroelasticity, and phase-field models) (Bahtiri et al., 2024).
• Stiff and High-Order Systems: Fourth-order and strain-gradient models of plasticity, ill-conditioned by conventional FEM, are directly addressed by PIDL frameworks, leveraging fully coupled residuals and mesh-free collocation (Tyagi et al., 2024).
• Generative and Probabilistic Modeling: Normalizing flows and diffusion-backbone networks extend PIDL-CF to uncertainty quantification, surrogate modeling, and the real-time generation of physically-constrained fields with targeted uncertainty—a key advantage over deterministic architectures (Mo et al., 2022, Qiu et al., 2024).
• Operational Limitations: Hyperbolic conservation laws with genuine shocks pose challenges due to non-smoothness; regularization through artificial diffusion, adaptive sampling, and specialized losses can mitigate but not eliminate these issues (Huang et al., 2023).

A flexible loss design, mesh-free implementation, and parallel optimization capabilities on modern hardware provide unmatched adaptability relative to traditional finite element or mesh-based solvers. However, the physics incorporated must align with the qualitative dynamics of the problem; mismatched physics-embedded operators can lead to degraded accuracy (Liu et al., 2022).

7. Outlook and Best Practices

Best practice in PIDL-CF involves:

• Precise encoding of all known boundary, initial, and physical constraints in the coupled loss, with preference for automatic differentiation-based implementation.
• Use of robust, scalable optimizers and mesh-free collocation approaches for high-dimensional domains.
• Emphasis on architectures that enforce symmetry, objectivity, and thermodynamic consistency by design, not only as after-the-fact penalties.
• Strategic application of regularization via physics constraints, particularly in data-scarce, stiff, or highly nonlinear settings.
• Comprehensive validation on synthetic and experimental datasets with careful ablation studies to quantify the contribution of physics-informed components.

In sum, PIDL-CF establishes a rigorous, extensible framework for hybrid scientific machine learning, enabling discovery, calibration, and uncertainty quantification in PDE-constrained systems that are inaccessible or ill-posed for conventional approaches (Tartakovsky et al., 2018, Bahtiri et al., 2024, Mo et al., 2022, Chen et al., 2020, Tyagi et al., 2024).