---
title: Physics-Embedded Neural ODEs (PENODE)
url: https://www.emergentmind.com/topics/physics-embedded-neural-odes-penode
type: topic
---

# Physics-Embedded Neural ODEs (PENODE)

Searching arXiv for recent and relevant papers on Physics-Embedded Neural ODEs and closely related methods.
arxiv_search query: "Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid" max_results: 10
arxiv_search query: "\"Physics-Embedded Neural ODEs\" OR PENODE OR \"physics-informed neural ODE\" OR \"physics-enhanced neural ODE\"" max_results: 10
Physics-Embedded Neural ODEs (PENODEs) are hybrid continuous-time models in which an ordinary differential equation retains selected mechanistic structure while neural components are introduced only where the governing dynamics are uncertain, incomplete, weakly trusted, or computationally inconvenient. Across the recent literature, this idea appears under several closely related names—“Physics-Embedded Neural ODEs,” “Physics-Informed Neural ODEs,” “Physics-enhanced Neural ODEs,” “Physics-based parameterized Neural ODEs,” “Physics-ENcoded Neural ODE,” and domain-specific variants such as Neural Modal ODEs—but the shared principle is the same: couple differentiable ODE evolution with explicit physical structure rather than learning the entire vector field as an unconstrained black box [2310.20079] [2508.02887] [2602.23670].

## 1. Conceptual scope and nomenclature

The PENODE family is not a single canonical architecture. In the tokamak setting, the method called “RomeroNNV” preserves two exact circuit-theoretic identities and replaces only the low-confidence closure for $\dot V$ with a neural network [2310.20079]. In antagonistic pneumatic artificial muscles, the hybrid model keeps parametric joint mechanics and pneumatic pressure dynamics while a neural force term captures antagonistic coupling and rate-dependent hysteresis [2602.23670]. In power electronics, PENODE combines continuous dynamics with an explicit event automaton, so that discrete switching is handled by hybrid-mode logic while the continuous vector field is decomposed into physics-based and neural-residual components [2508.02887].

The same literature also contains broader formulations. One line embeds first-principles mechanics through Lagrangian structure, with the neural network representing only non-conservative generalized forces [2005.14617]. Another uses a modal reduction viewpoint, where a finite element model supplies eigenmodes, eigenfrequencies, and modal damping, and the neural term acts as a residual in a latent modal ODE [2207.07883]. A separate interpretation of PENODE derives latent dynamics exactly from a known physical ODE by the chain rule after an invertible state lifting; there, no unknown latent dynamics are learned at all, and training acts only on the embedding so that latent trajectories evolve slowly [2605.06980].

This variation indicates that “physics-embedded” refers less to one fixed recipe than to a design doctrine. The doctrine is to decide which components should remain analytical, which should be soft penalties, which should be trainable residuals, and which should be transformed into a latent representation while preserving the original dynamics.

## 2. Recurrent mathematical patterns

A recurrent formulation is additive decomposition of the vector field,
\[
\dot x
=
f_{\mathrm{phys}}(x,u;\theta_{\mathrm{phys}})
+
f_{\mathrm{NN}}(x,u;\theta_{\mathrm{NN}}),
\]
used explicitly for antagonistic pneumatic artificial muscle dynamics and hybrid power electronics systems [2602.23670] [2508.02887]. In these cases, the analytical term carries low-order mechanics, gas laws, or linear circuit dynamics, and the neural term captures residual nonlinearities, coupling, hysteresis, or unmodeled effects.

A second pattern is partial equation replacement. In tokamak plasma inductance dynamics, the hybrid model defines
\[
\dot L_i=\frac{-2u-2V}{I_p},
\qquad
\dot I_p=\frac{2u+V}{L_i},
\qquad
\dot V=f_\theta(x,u),
\]
so that only the uncertain closure is learned, while the exact relations for $\dot L_i$ and $\dot I_p$ remain analytical [2310.20079]. The rocket-combustor PNODE follows the same logic at a finer granularity: the 0D stirred-tank reactor model is preserved, but deep networks predict sub-terms such as the heat-source function $\dot Q(t,\eta;\theta)$, pre-exponential factors $A_{f,j}(t,\eta;\theta)$, and activation energies $E_j(\eta;\theta)$ inside the governing equations [2302.08629].

A third pattern is structure-preserving mechanics. In the Lagrangian PINODE formulation,
\[
M(q)\,\ddot q + C(q,\dot q)\,\dot q + G(q)=Q^{NN}(q,\dot q,u;\theta),
\]
and therefore
\[
\dot x
=
\underbrace{
\begin{bmatrix}
\dot q\\
- M^{-1}(C\,\dot q+G)
\end{bmatrix}
}_{f_{\rm phys}(x)}
+
\underbrace{
\begin{bmatrix}
0\\
M^{-1}Q^{NN}(q,\dot q,u;\theta)
\end{bmatrix}
}_{f_{\rm NN}(x;\theta)}.
\]
The mass matrix, Coriolis terms, and conservative forces are hard-coded, and only the non-conservative contribution is approximated by a network [2005.14617].

A fourth pattern is soft physics regularization rather than hard structural embedding. MPINeuralODE defines a Neural-ODE backbone $\dot x=f_\theta(x)$, a pointwise residual
\[
R_{\mathrm{phys}}(x)=f_\theta(x)-f_{\mathrm{phys}}(x),
\]
and penalizes $\|R_{\mathrm{phys}}(x)\|^2$ on a collocation set built from predicted and true states. Its total loss combines data fitting, physics residual, continuity loss from multiple shooting, and $\ell_1$ regularization [2605.13305]. This is a PENODE-adjacent formulation because the mechanistic prior enters as a soft residual term rather than as an analytical state equation.

A fifth pattern appears in PDE operator learning. NODE-ONet evolves a latent state by
\[
\frac{dz(t)}{dt}=f_\theta\bigl(z(t),t;\mathcal P\bigr),
\]
with physical structure injected through projected conservation laws, frozen boundary coordinates, or embedded discrete operators such as $\mathbb L(a)$ [2510.15651]. This moves the PENODE idea from state-space system identification to surrogate solution operators for PDE families.

## 3. Mechanisms for embedding physics

The literature implements physics embedding through at least four distinct mechanisms.

**Hard analytical constraints** retain equations believed to be exact. The tokamak model keeps equations (1a)–(1b) from the Romero system unchanged, which the authors describe as “exact circuit-theoretic identities,” while only the least certain term is learned [2310.20079]. The cart-pole PINODE similarly preserves the Euler–Lagrange structure and thereby constrains the model to respect the Lagrangian formulation [2005.14617].

**Residual learning around a mechanistic backbone** augments a known model instead of replacing it. In the PAM formulation, the neural contribution acts only in the acceleration equation through a scalar learned antagonistic force, whereas pressure dynamics remain physics-based [2602.23670]. In power electronics, the continuous dynamics in each discrete mode are written as $A_zx+B_zu+f_{\mathrm{NN},z}(x,u;\theta_{\mathrm{NN}})$, with the known linear term carrying the dominant dynamics and the neural term correcting the remainder [2508.02887].

**Soft penalties and collocation residuals** impose mechanistic consistency statistically rather than identically. MPINeuralODE samples collocation states from both the predicted trajectory and the true trajectory, penalizes the vector-field mismatch $f_\theta-f_{\mathrm{phys}}$, and combines this with a Multiple-Initial-Condition multiple-shooting curriculum [2605.13305]. NODE-ONet includes an optional penalty
\[
\mathcal R_{\rm phys}(\theta)
=
\int_0^T \left\|\frac{dz}{dt}-\widetilde f_\theta(z,t)\right\|^2 dt
+
\lambda_{\rm bc}\sum_{b\in\partial\Omega}\bigl(Dz(t)_b-g(t)_b\bigr)^2
\]
when constraints are not structurally embedded [2510.15651].

**Coordinate transforms that preserve the original ODE exactly** define a different notion of embedding. In the physics-preserving acceleration framework, one lifts $x\in\mathbb R^n$ to $z\in\mathbb R^m$ via
\[
z=\phi(Ax), \qquad x=A^\dagger\phi^{-1}(z),
\]
and derives
\[
\dot z = J_\phi\!\bigl(\phi^{-1}(z)\bigr)\,A\,f\!\bigl(A^\dagger\phi^{-1}(z)\bigr)
\]
by exact application of the chain rule [2605.06980]. Here the latent-space ODE is not a learned surrogate for unknown physics; the embedding is trained so that the latent dynamics become slow and can be integrated more cheaply.

In hybrid systems, the embedding can also include **discrete mode logic**. The power-electronics PENODE defines a hybrid automaton
\[
\mathcal H=(\mathcal Z,\mathcal X,F,\mathcal E,G,\Delta,R),
\]
with guard-triggered transitions, optional reset map, and mode-dependent continuous vector fields [2508.02887]. This is notable because the physical prior is not only differential but also event-structural.

## 4. Training, integration, and optimization

Training procedures are as heterogeneous as the model architectures. Solver-in-the-loop backpropagation remains common. The tokamak PENODE uses JAX + Equinox for model definition, Diffrax for differentiable ODE solvers, cubic Hermite splines with backward-difference slopes for causal control interpolation, AdamW with initial learning rate $5\times10^{-3}$ and exponential decay factor 0.995, and early stopping when validation loss does not improve over 1000 epochs [2310.20079]. The PAM formulation uses the Tsit5 ODE solver with adjoint sensitivity, Adam with initial learning rate $1\times10^{-2}$ decayed by 0.95 on plateau, early stopping with patience 100 epochs, and a staged curriculum that begins with auxiliary viscous damping $B\dot x$ and progressively removes it [2602.23670].

Fixed-step classical integration also appears. The Lagrangian PINODE is trained end-to-end through a fixed-step Runge–Kutta 4 integrator using TensorFlow reverse-mode autodiff [2005.14617]. The rocket-combustor PNODE uses fourth-order Runge–Kutta built in ADCME and L-BFGS-B on the network and heat-normalization parameters [2302.08629]. In contrast, the physics-preserving latent-space formulation does not learn the dynamics from trajectories; it optimizes the lift-and-coupling map using a Jacobian-norm loss
\[
L
=
\frac{1}{Nk}\sum_{i=1}^N\sum_{j=1}^k
\|J_{\dot z}(z_i)v_j\|_2^2
\]
so that latent evolution becomes slow in all directions [2605.06980].

Multiple-shooting is a recurrent stabilization device. The tokamak work reports multiple shooting with group size 10 and zero continuity penalty, and states that it was found to stabilize long-horizon training [2310.20079]. MPINeuralODE uses a more explicit multiple-shooting curriculum: each epoch samples $m=128$ initial conditions from a mixed sampler, partitions the horizon into $K=4$ equal subintervals, integrates each segment as an independent initial-value problem via detach, and adds a soft continuity loss
\[
L_{\mathrm{cont}}
=
\frac{1}{K-1}\sum_{k=1}^{K-1}
\|\hat x_{k-1}(\mathrm{end})-x_{\mathrm{true},k}(\mathrm{start})\|_2^2
\]
with default weight $\lambda_{\mathrm{cont}}=1$ [2605.13305].

A separate training paradigm reformulates PeNODE learning as a dynamic optimization problem. Direct collocation with flipped Legendre–Gauss–Radau points converts the full hybrid model and trajectory fit into a large-scale nonlinear program, solved by Ipopt with exact Jacobian/Hessian sparsity, MUMPS linear algebra, and OpenMP-parallel callback evaluation [2505.03552]. On the Quarter Vehicle Model, even the naive strategy converges stably in $\ll 7$ min versus 4.5 h for the ODE-solver-based baseline, and a rational-surrogate variant trains in 34.96 s [2505.03552]. This establishes that the numerical method used to train a PENODE can be as consequential as the embedding itself.

## 5. Representative domains and reported performance

The application space of PENODEs is unusually broad, spanning plasma control, soft robotics, structural dynamics, power electronics, combustion, latent-space acceleration of ODE simulation, and PDE operator learning.

| Domain | Embedding strategy | Representative result |
|---|---|---|
| Tokamak plasma inductance dynamics | Exact circuit equations + neural closure for $\dot V$ | RomeroNNV: $3.50\%\pm3.70\%$ on $L_i$, $1.97\%\pm1.90\%$ on $I_p$ [2310.20079] |
| Hybrid power electronics EDTs | Event automaton + physics-based mode dynamics + neural residual | 75% reduction in neuron count; 6 $\mu$s per step at $1e^{-4}$ error tolerance [2508.02887] |
| Antagonistic PAM dynamics | Physics-based mechanics and gas dynamics + learned antagonistic force | Mean $R^2=0.88$ across 225 conditions; stiffness control across 126–176 N/mm [2602.23670] |
| Lotka–Volterra system learning | Soft physics residual + MIC multiple shooting | MPINeuralODE OOS MSE 15.12, 26% reduction over baseline LV_NN [2605.13305] |
| ODE simulation acceleration | Exact latent dynamics derived by chain rule | 3×–20× fewer function calls for the same accuracy [2605.06980] |
| Laser ignition in a rocket combustor | 0D reactor model + learned $\dot Q$, $A_f$, and $E$ terms | Final-$T$ mean relative error 40.36% with 100 training samples, versus 88.76% for NN [2302.08629] |

These results illustrate several recurring empirical claims.

First, hybridization often improves generalization relative to both pure physics and pure neural baselines. In the tokamak study, the hybrid PENODE outperforms both the pure physics Romero model and the pure neural ODE MlpODE on end-of-pulse test error for both internal inductance and plasma current, while the pure NN model is reported to exhibit severe overfitting and “double-descent” behavior on validation [2310.20079]. In the power-electronics study, PENODE is reported to achieve 0.2% current-tracking MRE in the white-box case, 0.4% out-of-domain MRE in the gray-box case, and 1.0% MRE with 25% data in the black-box case, with additional benefits in FPGA deployment and closed-loop MPC [2508.02887].

Second, embedding can improve long-horizon or off-support behavior. MPINeuralODE is explicitly motivated by the observation that standard Neural ODEs often fit training trajectories while generalizing poorly to unseen initial conditions and long horizons. On held-out Lotka–Volterra initial conditions over $[0,30]$, MPINeuralODE achieves the lowest out-of-sample and long-horizon MSE among data-driven methods, with a 26% reduction over the baseline Neural ODE and relative Hamiltonian drift 0.943, essentially matching the PINN ablation on Hamiltonian drift [2605.13305]. In the PDE context, NODE-ONet reports stable extrapolation beyond the training horizon, including prediction up to $t=2T$ with error $<10^{-2}$ on the nonlinear diffusion–reaction example and error $\mathcal O(10^{-2})$ on Navier–Stokes prediction extended from $[0,10]$ to $[0,20]$ [2510.15651].

Third, the hybrid split can reduce model size or sample burden. The power-electronics PENODE reports that the neural residual requires only $\approx25\%$ of the neurons of a black-box NODE, corresponding to a 75% reduction in neuron count [2508.02887]. The PAM study attributes sample efficiency to reducing the network’s burden to antagonistic coupling and hysteresis, and states that the required training set shrinks to 29 of 225 datasets [2602.23670]. The rocket-combustor PNODE reports sharper ignition-boundary prediction with limited data than a fully connected neural network or kernel ridge regression [2302.08629].

## 6. Misconceptions, limitations, and active directions

A common misconception is that PENODEs always impose physics as an exact conservation law. The reported literature does not support that claim. In MPINeuralODE, “No symplectic integrator or explicit Hamiltonian network” is used; the conservation law is enforced only softly via $L_{\mathrm{phys}}$ [2605.13305]. In PDE operator learning, constraints may be either structurally built in or softly penalized, depending on the design [2510.15651]. This indicates that physical plausibility in PENODEs ranges from hard invariance to regularized consistency.

A second misconception is that PENODE means “learn the whole ODE, then add a physics loss.” Several successful instantiations do the opposite. The tokamak PENODE learns only the uncertain closure $\dot V$ [2310.20079]. The Lagrangian PINODE learns only non-conservative forces [2005.14617]. The PAM model learns only a force component in one state derivative while keeping the remaining state equations analytical [2602.23670]. The acceleration-oriented PENODE learns no unknown dynamics in latent space at all, because the latent ODE is derived exactly by the chain rule [2605.06980].

The literature also makes clear that benefits are conditional rather than automatic. PINODE requires partial knowledge of the system, specifically accurate $M$, $C$, and $G$ terms, and is still sensitive to solver choice and step size [2005.14617]. Power-electronics PENODE depends on a correct event automaton and cloud-to-edge quantization and deployment stack [2508.02887]. Direct-collocation training addresses stability and runtime limitations of solver-based training, but at the cost of solving a large sparse nonlinear program [2505.03552]. These examples suggest that the “physics” inserted into a PENODE can reduce variance and improve extrapolation only when the inserted structure is itself well matched to the system.

Reported future directions are correspondingly diverse. The direct-collocation line outlines integration into OpenModelica to enable training of Neural DAEs natively [2505.03552]. Neural Modal ODEs explicitly propose Bayesian neural networks for uncertainty quantification, strongly nonlinear systems through nonlinear normal modes, PDE-to-latent-ODE constructions via spectral or Galerkin reduction, and multi-physics coupling through block-diagonal physics matrices [2207.07883]. NODE-ONet emphasizes flexibility across encoders, decoders, and related PDE families [2510.15651]. The aggregate direction of the field is therefore not toward a single standardized PENODE, but toward a broad class of hybrid continuous-time models in which mechanistic structure, numerical solvers, and trainable residuals are co-designed for the target scientific domain.

Source: https://www.emergentmind.com/topics/physics-embedded-neural-odes-penode