Physics-Informed Energy-Based Model (PI-EBM)

• PI-EBM is a machine learning paradigm that combines physics-based constraints with energy or Lyapunov loss components to enforce structure preservation and stability.
• It incorporates explicit energy metrics into the loss function, improving noise robustness and ensuring adherence to physical laws in diverse tasks like PDE solving and inverse problems.
• Empirical results demonstrate that PI-EBMs outperform traditional PINNs in error reduction, scientific interpretability, and robustness in applications such as seismic prediction and image classification.

A Physics-Informed Energy-Based Model (PI-EBM) is a machine learning paradigm uniting physics-based constraints and energy-based modeling frameworks to represent and solve forward/inverse problems governed by physical laws, incorporating domain-specific energy, stability, or noise models directly into the network architecture and loss functions. Recent research demonstrates that PI-EBMs surpass traditional PINN approaches in physical structure preservation, robustness to non-Gaussian measurement errors, and scientific interpretability in multi-task and classification domains (Chu et al., 2024, Pilar et al., 2022, Kriuk et al., 5 Jan 2026).

1. Foundations and Formal Definition

PI-EBMs extend physics-informed neural networks (PINNs) by integrating explicit energy or Lyapunov loss components, or modeling measurement noise with energy-based models (EBMs) for maximum-likelihood inference.

Structure-Preserving Loss Construction

Given a target PDE (e.g., Allen–Cahn), the total loss consists of standard PINN residuals (equation, boundary, initial conditions) plus a structure-preserving penalty that enforces physical constraints such as energy dissipation: $$L_{\rm total} = \lambda_1 L_{\rm eqn} + \lambda_2 L_{\rm bnd} + \lambda_3 L_{\rm ini} + \lambda_4 L_{\rm strc}$$ where $L_{\rm strc}$ penalizes upward drift in the energy functional $J$: $$L_{\rm strc} = \frac{1}{N_e} \sum_{i=1}^{N_e} \| \operatorname{ReLU}\left(\frac{d}{dt} J(\hat u(\cdot, t_i))\right) \|_2$$ Automatic differentiation is employed to construct $J$ and its time derivative.

Physics-Informed Noise Modeling

When physical observations are contaminated by unknown, non-Gaussian noise, PI-EBMs employ a neural EBM to learn the likelihood model for residuals: $$p_{\rm noise}(r | \theta_{\rm EBM}) = \frac{\exp(h(r; \theta_{\rm EBM}))}{Z(\theta_{\rm EBM})}$$ yielding the data-fitting loss: $$\mathcal{L}_{\rm data}^{\rm EBM} = -\frac{1}{N_d}\sum_{i=1}^{N_d} \log p_{\rm noise}(r^i;\theta_{\rm EBM})$$ The final objective combines this EBM likelihood with the physics loss, restoring consistency and robustness: $$\mathcal{L}_{\rm tot} = \mathcal{L}_{\rm data}^{\rm EBM} + \omega \mathcal{L}_{\rm PDE}$$ (Pilar et al., 2022).

2. Integration of Physics and Energy Constraints

The core advancement of PI-EBMs is the direct enforcement of physical principles within the learning process. The physical structure is encoded either as an energy/entropy functional (for PDEs and dynamical systems) or as Lyapunov-type stability criteria (for classification, ODEs, and learning tasks):

• Energy Conservation/Dissipation: PDE solvers penalize violations of energy decay/constancy by differentiating the relevant functionals and including them in the training loss (Chu et al., 2024).
• Lyapunov Stability: For neural ODEs or classification, the network learns a Lyapunov function $V(\mathbf u)$ and projects dynamics such that $\nabla V^\top F(\mathbf u) \leq -c V(\mathbf u)$, correcting trajectories toward stability (Chu et al., 2024).
• Seismological Constraints: In multi-task geophysical modeling, laws such as Gutenberg–Richter, Omori–Utsu, and Bath’s are parameterized as neural constraints with learnable physical parameters. These are regularized directly in the overall objective (Kriuk et al., 5 Jan 2026).

3. Architecture and Training Procedures

PI-EBMs utilize diverse architectures tailored to the task domain:

• PDEs and Inverse Problems: Fully-connected networks with multiple hidden layers; training alternates standard PINN losses and structure-preserving penalties.
• Classification/Image Domains: ResNet-18 trunk, Input-Convex Neural Network (ICNN) for Lyapunov computation; alternating forward-inverse optimization steps with learning rate scheduling (Chu et al., 2024).
• Seismic Event Prediction: Multi-scale convolutional grid encoders, local event feature MLPs, fusion stages, and scalar energy heads. Physics constraints are introduced as differentiable, learnable layers, allowing direct backpropagation into scientific parameters (Kriuk et al., 5 Jan 2026).

Optimization incorporates Adam/L-BFGS, mini-batch sampling over collocation/data points, and sometimes staged training to bootstrap neural components before joint fine-tuning. For EBMs modeling 1D noise, partition functions are computed via numerical quadrature, avoiding MCMC complexity. In higher dimensions, Langevin sampling may be employed (Pilar et al., 2022).

4. Empirical Performance and Scientific Interpretability

PI-EBMs demonstrate considerable improvements over baselines across domains:

• Allen–Cahn PDE: SP-PINN halves test error versus vanilla PINN, ensures numerically correct energy behavior, and is computationally efficient compared to discrete variational structure-preserving methods (Chu et al., 2024).
• Image Classification under Adversarial Perturbations: The Lyapunov-augmented model improves adversarial robustness by up to 21.8 percentage points in CIFAR-10 and up to 29 points under adversarial training, with clean accuracy matching vanilla approaches (Chu et al., 2024).
• Seismic Multi-Task Prediction: POSEIDON achieves state-of-the-art aftershock F1 (0.762), tsunami F1 (0.407), and foreshock F1 (0.556), outperforming tree/CNN baselines. Learned physical parameters converge to interpretable, domain-consistent values (Gutenberg–Richter $b=0.752$, Omori–Utsu $p=0.835$, $c=0.1948$ days, Bath’s gap $\Delta M=-0.130$), confirming scientific fidelity (Kriuk et al., 5 Jan 2026).
• Noise-Robust Inverse Problems: In parameter estimation for ODEs and Navier–Stokes, PI-EBMs sharply outperform LS-based PINNs, correctly matching the underlying noise distributions and restoring unbiased parameter recovery (Pilar et al., 2022).

5. Extension to Novel Domains and Downstream Tasks

PI-EBMs’ modular penalty construction and energy/Lyapunov integration allow broad applicability:

• The structure-preserving loss can extend to other conservative PDEs (KdV, nonlinear Schrödinger), or dissipative systems by sign reversal.
• In inverse problems, other invariants (e.g., mass, momentum) may replace energy as constraints.
• Any Neural ODE-based architecture is compatible with Lyapunov-stability projection.
• Stability constraints confer robustness to tasks where solution stability under perturbation is essential (generative modeling, reinforcement learning, robust feature extraction).
• Joint networks can couple PDE solving and robust feature learning within a unified PI-EBM framework (Chu et al., 2024).

A plausible implication is that PI-EBMs may facilitate the development of interpretable scientific deep learning tools robust to out-of-distribution errors and suitable for high-stakes downstream applications.

6. Robustness, Limitations, and Open Problems

PI-EBMs address key deficiencies in standard PINNs, notably nonphysical behavior, inefficiency, and sensitivity to outlier noise:

• Robustness to Measurement Error: EBMs model the true noise distribution, preventing statistical bias and inconsistency, especially under heavy-tailed or multimodal error.
• Structure Preservation: Explicit energy/Lyapunov penalties enforce fundamental physical laws, so learned solutions display genuine scientific validity.
• Limitations: For highly complex, multi-dimensional noise, EBM partition function estimation may require advanced sampling. Structure-preserving constraints are contingent on available prior knowledge of system invariants or stability properties.

The joint maximum-likelihood estimators produced by PI-EBMs offer formal guarantees of consistency as data volume increases, a significant theoretical advancement over previous PINN frameworks (Pilar et al., 2022). Open challenges remain in scalable partition function estimation, automated constraint discovery, and generalization to systems lacking well-characterized invariants.

7. Key Papers, Resources, and Future Directions

• Structure-Preserving PINNs: "Structure-Preserving Physics-Informed Neural Networks With Energy or Lyapunov Structure" (Chu et al., 2024).
• Noise-Robust PI-EBMs: "Physics-informed Neural Networks with Unknown Measurement Noise" (Pilar & Wahlström) (Pilar et al., 2022).
• Geophysical Multi-Task PI-EBMs: "POSEIDON: Physics-Optimized Seismic Energy Inference and Detection Operating Network" (Kriuk et al., 5 Jan 2026).

The Poseidon dataset for seismic research, featuring energy features and standardized benchmarks, is publicly available to support further PI-EBM evaluation (Kriuk et al., 5 Jan 2026). Future research may expand PI-EBM principles to data assimilation, uncertainty quantification, and automated physics discovery across scientific disciplines.