Physics-Embedded Neural ODEs (PENODE)

• Physics-Embedded Neural ODEs (PENODEs) are hybrid continuous-time models that integrate established physical laws with neural network residuals to capture uncertain dynamics.
• They employ methodologies such as additive residual decomposition, partial equation replacement, and soft physics regularization to balance analytic structure with data-driven learning.
• PENODEs enhance generalization, sample efficiency, and computational performance across diverse applications including plasma control, robotics, and power electronics.

Searching arXiv for recent and relevant papers on Physics-Embedded Neural ODEs and closely related methods. arxiv_search query: "^^^^^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^^^^^" max_results: ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ arxiv_search query: "^^^^^^^^4\4 Neural ODEs^^^^^^^^4\4 OR PENODE OR ^^^^^^^^4\4 neural ODE^^^^^^^^4\4 OR ^^^^^^^^4\4 neural ODE^^^^^^^^4\4 max_results: ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ Physics-Embedded Neural ODEs (PENODEs) are hybrid continuous-time models in which an ordinary differential equation retains selected mechanistic structure while neural components are introduced only where the governing dynamics are uncertain, incomplete, weakly trusted, or computationally inconvenient. Across the recent literature, this idea appears under several closely related names—“Physics-Embedded Neural ODEs,” “Physics-Informed Neural ODEs,” “Physics-enhanced Neural ODEs,” “Physics-based parameterized Neural ODEs,” “Physics-ENcoded Neural ODE,” and domain-specific variants such as Neural Modal ODEs—but the shared principle is the same: couple differentiable ODE evolution with explicit physical structure rather than learning the entire vector field as an unconstrained black box (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&, &&&^^^^4\4^^^^&&&, &&&^^^^4 OR PENODE OR \4^^^^&&&).

^^^^4\4^^^^. Conceptual scope and nomenclature

The PENODE family is not a single canonical architecture. In the tokamak setting, the method called “RomeroNNV” preserves two exact circuit-theoretic identities and replaces only the low-confidence closure for PRESERVED_PLACEHOLDER_^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ with a neural network (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). In antagonistic pneumatic artificial muscles, the hybrid model keeps parametric joint mechanics and pneumatic pressure dynamics while a neural force term captures antagonistic coupling and rate-dependent hysteresis (&&&^^^^4 OR PENODE OR \4^^^^&&&). In power electronics, PENODE combines continuous dynamics with an explicit event automaton, so that discrete switching is handled by hybrid-mode logic while the continuous vector field is decomposed into physics-based and neural-residual components (&&&^^^^4\4^^^^&&&).

The same literature also contains broader formulations. One line embeds first-principles mechanics through Lagrangian structure, with the neural network representing only non-conservative generalized forces (Roehrl et al., 2020). Another uses a modal reduction viewpoint, where a finite element model supplies eigenmodes, eigenfrequencies, and modal damping, and the neural term acts as a residual in a latent modal ODE (Lai et al., 2022). A separate interpretation of PENODE derives latent dynamics exactly from a known physical ODE by the chain rule after an invertible state lifting; there, no unknown latent dynamics are learned at all, and training acts only on the embedding so that latent trajectories evolve slowly (Tagg et al., 7 May 2026).

This variation indicates that “physics-embedded” refers less to one fixed recipe than to a design doctrine. The doctrine is to decide which components should remain analytical, which should be soft penalties, which should be trainable residuals, and which should be transformed into a latent representation while preserving the original dynamics.

^^^^4 OR PENODE OR \4^^^^. Recurrent mathematical patterns

A recurrent formulation is additive decomposition of the vector field,

PRESERVED_PLACEHOLDER_^^^^4\4^^^^

used explicitly for antagonistic pneumatic artificial muscle dynamics and hybrid power electronics systems (&&&^^^^4 OR PENODE OR \4^^^^&&&, &&&^^^^4\4^^^^&&&). In these cases, the analytical term carries low-order mechanics, gas laws, or linear circuit dynamics, and the neural term captures residual nonlinearities, coupling, hysteresis, or unmodeled effects.

A second pattern is partial equation replacement. In tokamak plasma inductance dynamics, the hybrid model defines

PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^

so that only the uncertain closure is learned, while the exact relations for PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^ and $\dot I_p$ remain analytical (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). The rocket-combustor PNODE follows the same logic at a finer granularity: the ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^D stirred-tank reactor model is preserved, but deep networks predict sub-terms such as the heat-source function $\dot Q(t,\eta;\theta)$, pre-exponential factors $A_{f,j}(t,\eta;\theta)$, and activation energies $E_j(\eta;\theta)$ inside the governing equations (&&&^^^^4\4 OR PENODE OR \4^^^^&&&).

A third pattern is structure-preserving mechanics. In the Lagrangian PINODE formulation,

$$M(q)\,\ddot q + C(q,\dot q)\,\dot q + G(q)=Q^{NN}(q,\dot q,u;\theta),$$

and therefore

$$\dot x = \underbrace{ \begin{bmatrix} \dot q\ - M^{-1}(C\,\dot q+G) \end{bmatrix} }_{f_{\rm phys}(x)} + \underbrace{ \begin{bmatrix} 0\ M^{-1}Q^{NN}(q,\dot q,u;\theta) \end{bmatrix} }_{f_{\rm NN}(x;\theta)}.$$

The mass matrix, Coriolis terms, and conservative forces are hard-coded, and only the non-conservative contribution is approximated by a network (Roehrl et al., 2020).

A fourth pattern is soft physics regularization rather than hard structural embedding. MPINeuralODE defines a Neural-ODE backbone PRESERVED_PLACEHOLDER_^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^, a pointwise residual

PRESERVED_PLACEHOLDER_^^^^4\4\4^^^^

and penalizes PRESERVED_PLACEHOLDER_^^^^4\4 OR PENODE OR \4^^^^ on a collocation set built from predicted and true states. Its total loss combines data fitting, physics residual, continuity loss from multiple shooting, and PRESERVED_PLACEHOLDER_^^^^4\4 OR \4^^^^ regularization (&&&^^^^4\44^^^^&&&). This is a PENODE-adjacent formulation because the mechanistic prior enters as a soft residual term rather than as an analytical state equation.

A fifth pattern appears in PDE operator learning. NODE-ONet evolves a latent state by

PRESERVED_PLACEHOLDER_^^^^4\44^^^^

with physical structure injected through projected conservation laws, frozen boundary coordinates, or embedded discrete operators such as PRESERVED_PLACEHOLDER_^^^^4\4^^^^5 (&&&^^^^4\4^^^^5&&&). This moves the PENODE idea from state-space system identification to surrogate solution operators for PDE families.

^^^^4 OR \4^^^^. Mechanisms for embedding physics

The literature implements physics embedding through at least four distinct mechanisms.

Hard analytical constraints retain equations believed to be exact. The tokamak model keeps equations (^^^^4\4^^^^a)–(^^^^4\4 from the Romero system unchanged, which the authors describe as “exact circuit-theoretic identities,” while only the least certain term is learned (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). The cart-pole PINODE similarly preserves the Euler–Lagrange structure and thereby constrains the model to respect the Lagrangian formulation (Roehrl et al., 2020).

Residual learning around a mechanistic backbone augments a known model instead of replacing it. In the PAM formulation, the neural contribution acts only in the acceleration equation through a scalar learned antagonistic force, whereas pressure dynamics remain physics-based (&&&^^^^4 OR PENODE OR \4^^^^&&&). In power electronics, the continuous dynamics in each discrete mode are written as PRESERVED_PLACEHOLDER_^^^^4\4^^^^6, with the known linear term carrying the dominant dynamics and the neural term correcting the remainder (&&&^^^^4\4^^^^&&&).

Soft penalties and collocation residuals impose mechanistic consistency statistically rather than identically. MPINeuralODE samples collocation states from both the predicted trajectory and the true trajectory, penalizes the vector-field mismatch PRESERVED_PLACEHOLDER_^^^^4\4^^^^7, and combines this with a Multiple-Initial-Condition multiple-shooting curriculum (&&&^^^^4\44^^^^&&&). NODE-ONet includes an optional penalty

PRESERVED_PLACEHOLDER_^^^^4\4^^^^8

when constraints are not structurally embedded (&&&^^^^4\4^^^^5&&&).

Coordinate transforms that preserve the original ODE exactly define a different notion of embedding. In the physics-preserving acceleration framework, one lifts PRESERVED_PLACEHOLDER_^^^^4\4^^^^9 to PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ via

PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4\4^^^^

and derives

PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4 OR PENODE OR \4^^^^

by exact application of the chain rule (Tagg et al., 7 May 2026). Here the latent-space ODE is not a learned surrogate for unknown physics; the embedding is trained so that the latent dynamics become slow and can be integrated more cheaply.

In hybrid systems, the embedding can also include discrete mode logic. The power-electronics PENODE defines a hybrid automaton

PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4 OR \4^^^^

with guard-triggered transitions, optional reset map, and mode-dependent continuous vector fields (&&&^^^^4\4^^^^&&&). This is notable because the physical prior is not only differential but also event-structural.

4. Training, integration, and optimization

Training procedures are as heterogeneous as the model architectures. Solver-in-the-loop backpropagation remains common. The tokamak PENODE uses JAX + Equinox for model definition, Diffrax for differentiable ODE solvers, cubic Hermite splines with backward-difference slopes for causal control interpolation, AdamW with initial learning rate PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \44^^^^ and exponential decay factor ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.995, and early stopping when validation loss does not improve over ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ epochs (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). The PAM formulation uses the Tsit5 ODE solver with adjoint sensitivity, Adam with initial learning rate PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^5 decayed by ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.95 on plateau, early stopping with patience ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ epochs, and a staged curriculum that begins with auxiliary viscous damping PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^6 and progressively removes it (&&&^^^^4 OR PENODE OR \4^^^^&&&).

Fixed-step classical integration also appears. The Lagrangian PINODE is trained end-to-end through a fixed-step Runge–Kutta 4 integrator using TensorFlow reverse-mode autodiff (Roehrl et al., 2020). The rocket-combustor PNODE uses fourth-order Runge–Kutta built in ADCME and L-BFGS-B on the network and heat-normalization parameters (&&&^^^^4\4 OR PENODE OR \4^^^^&&&). In contrast, the physics-preserving latent-space formulation does not learn the dynamics from trajectories; it optimizes the lift-and-coupling map using a Jacobian-norm loss

PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^7

so that latent evolution becomes slow in all directions (Tagg et al., 7 May 2026).

Multiple-shooting is a recurrent stabilization device. The tokamak work reports multiple shooting with group size ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ and zero continuity penalty, and states that it was found to stabilize long-horizon training (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). MPINeuralODE uses a more explicit multiple-shooting curriculum: each epoch samples PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^8 initial conditions from a mixed sampler, partitions the horizon into PRESERVED_PLACEHOLDER_^^^^4 OR PENODE OR \4^^^^9 equal subintervals, integrates each segment as an independent initial-value problem via detach, and adds a soft continuity loss

PRESERVED_PLACEHOLDER_^^^^4 OR \4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^

with default weight PRESERVED_PLACEHOLDER_^^^^4 OR \4\4^^^^ (&&&^^^^4\44^^^^&&&).

A separate training paradigm reformulates PeNODE learning as a dynamic optimization problem. Direct collocation with flipped Legendre–Gauss–Radau points converts the full hybrid model and trajectory fit into a large-scale nonlinear program, solved by Ipopt with exact Jacobian/Hessian sparsity, MUMPS linear algebra, and OpenMP-parallel callback evaluation (&&&^^^^4 OR \4\4^^^^&&&). On the Quarter Vehicle Model, even the naive strategy converges stably in PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR PENODE OR \4^^^^ min versus 4.5 h for the ODE-solver-based baseline, and a rational-surrogate variant trains in ^^^^4 OR \44^^^^.96 s (&&&^^^^4 OR \4\4^^^^&&&). This establishes that the numerical method used to train a PENODE can be as consequential as the embedding itself.

5. Representative domains and reported performance

The application space of PENODEs is unusually broad, spanning plasma control, soft robotics, structural dynamics, power electronics, combustion, latent-space acceleration of ODE simulation, and PDE operator learning.

Domain	Embedding strategy	Representative result
Tokamak plasma inductance dynamics	Exact circuit equations + neural closure for PRESERVED_PLACEHOLDER_^^^^4 OR \4 OR \4^^^^	RomeroNNV: PRESERVED_PLACEHOLDER_^^^^4 OR \44^^^^ on PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^5, PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^6 on PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^7 (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&)
Hybrid power electronics EDTs	Event automaton + physics-based mode dynamics + neural residual	75% reduction in neuron count; 6 PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^8s per step at PRESERVED_PLACEHOLDER_^^^^4 OR \4^^^^9 error tolerance (&&&^^^^4\4^^^^&&&)
Antagonistic PAM dynamics	Physics-based mechanics and gas dynamics + learned antagonistic force	Mean $\dot I_p$^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ across ^^^^4 OR PENODE OR \4 OR PENODE OR \4^^^^5 conditions; stiffness control across ^^^^4\4 OR PENODE OR \4^^^^6–^^^^4\4^^^^76 N/mm (&&&^^^^4 OR PENODE OR \4^^^^&&&)
Lotka–Volterra system learning	Soft physics residual + MIC multiple shooting	MPINeuralODE OOS MSE ^^^^4\4^^^^5.^^^^4\4 OR PENODE OR \4^^^^, ^^^^4 OR PENODE OR \4^^^^6% reduction over baseline LV_NN (&&&^^^^4\44^^^^&&&)
ODE simulation acceleration	Exact latent dynamics derived by chain rule	^^^^4 OR \4^^^^×–^^^^4 OR PENODE OR \4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^× fewer function calls for the same accuracy (Tagg et al., 7 May 2026)
Laser ignition in a rocket combustor	^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^D reactor model + learned $\dot I_p$^^^^4\4^^^^, $\dot I_p$^^^^4 OR PENODE OR \4^^^^, and $\dot I_p$^^^^4 OR \4^^^^ terms	Final-$\dot I_p$4 mean relative error 4^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.^^^^4 OR \4^^^^6% with ^^^^4\4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ training samples, versus 88.76% for NN (&&&^^^^4\4 OR PENODE OR \4^^^^&&&)

These results illustrate several recurring empirical claims.

First, hybridization often improves generalization relative to both pure physics and pure neural baselines. In the tokamak study, the hybrid PENODE outperforms both the pure physics Romero model and the pure neural ODE MlpODE on end-of-pulse test error for both internal inductance and plasma current, while the pure NN model is reported to exhibit severe overfitting and “double-descent” behavior on validation (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). In the power-electronics study, PENODE is reported to achieve ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.^^^^4 OR PENODE OR \4^^^^% current-tracking MRE in the white-box case, ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.4% out-of-domain MRE in the gray-box case, and ^^^^4\4^^^^.^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^% MRE with ^^^^4 OR PENODE OR \4^^^^5% data in the black-box case, with additional benefits in FPGA deployment and closed-loop MPC (&&&^^^^4\4^^^^&&&).

Second, embedding can improve long-horizon or off-support behavior. MPINeuralODE is explicitly motivated by the observation that standard Neural ODEs often fit training trajectories while generalizing poorly to unseen initial conditions and long horizons. On held-out Lotka–Volterra initial conditions over $\dot I_p$5, MPINeuralODE achieves the lowest out-of-sample and long-horizon MSE among data-driven methods, with a ^^^^4 OR PENODE OR \4^^^^6% reduction over the baseline Neural ODE and relative Hamiltonian drift ^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^.94^^^^4 OR \4^^^^, essentially matching the PINN ablation on Hamiltonian drift (&&&^^^^4\44^^^^&&&). In the PDE context, NODE-ONet reports stable extrapolation beyond the training horizon, including prediction up to $\dot I_p$6 with error $\dot I_p$7 on the nonlinear diffusion–reaction example and error $\dot I_p$8 on Navier–Stokes prediction extended from $\dot I_p$9 to $\dot Q(t,\eta;\theta)$^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^ (&&&^^^^4\4^^^^5&&&).

Third, the hybrid split can reduce model size or sample burden. The power-electronics PENODE reports that the neural residual requires only $\dot Q(t,\eta;\theta)$^^^^4\4^^^^ of the neurons of a black-box NODE, corresponding to a 75% reduction in neuron count (&&&^^^^4\4^^^^&&&). The PAM study attributes sample efficiency to reducing the network’s burden to antagonistic coupling and hysteresis, and states that the required training set shrinks to ^^^^4 OR PENODE OR \4^^^^9 of ^^^^4 OR PENODE OR \4 OR PENODE OR \4^^^^5 datasets (&&&^^^^4 OR PENODE OR \4^^^^&&&). The rocket-combustor PNODE reports sharper ignition-boundary prediction with limited data than a fully connected neural network or kernel ridge regression (&&&^^^^4\4 OR PENODE OR \4^^^^&&&).

6. Misconceptions, limitations, and active directions

A common misconception is that PENODEs always impose physics as an exact conservation law. The reported literature does not support that claim. In MPINeuralODE, “No symplectic integrator or explicit Hamiltonian network” is used; the conservation law is enforced only softly via $\dot Q(t,\eta;\theta)$^^^^4 OR PENODE OR \4^^^^ (&&&^^^^4\44^^^^&&&). In PDE operator learning, constraints may be either structurally built in or softly penalized, depending on the design (&&&^^^^4\4^^^^5&&&). This indicates that physical plausibility in PENODEs ranges from hard invariance to regularized consistency.

A second misconception is that PENODE means “learn the whole ODE, then add a physics loss.” Several successful instantiations do the opposite. The tokamak PENODE learns only the uncertain closure $\dot Q(t,\eta;\theta)$^^^^4 OR \4^^^^ (&&&^^^^4Physics-Embedded Neural ODEs PENODE neural ODE physics embedded hybrid4^^^^&&&). The Lagrangian PINODE learns only non-conservative forces (Roehrl et al., 2020). The PAM model learns only a force component in one state derivative while keeping the remaining state equations analytical (&&&^^^^4 OR PENODE OR \4^^^^&&&). The acceleration-oriented PENODE learns no unknown dynamics in latent space at all, because the latent ODE is derived exactly by the chain rule (Tagg et al., 7 May 2026).

The literature also makes clear that benefits are conditional rather than automatic. PINODE requires partial knowledge of the system, specifically accurate $\dot Q(t,\eta;\theta)$4, $\dot Q(t,\eta;\theta)$5, and $\dot Q(t,\eta;\theta)$6 terms, and is still sensitive to solver choice and step size (Roehrl et al., 2020). Power-electronics PENODE depends on a correct event automaton and cloud-to-edge quantization and deployment stack (&&&^^^^4\4^^^^&&&). Direct-collocation training addresses stability and runtime limitations of solver-based training, but at the cost of solving a large sparse nonlinear program (&&&^^^^4 OR \4\4^^^^&&&). These examples suggest that the “physics” inserted into a PENODE can reduce variance and improve extrapolation only when the inserted structure is itself well matched to the system.

Reported future directions are correspondingly diverse. The direct-collocation line outlines integration into OpenModelica to enable training of Neural DAEs natively (&&&^^^^4 OR \4\4^^^^&&&). Neural Modal ODEs explicitly propose Bayesian neural networks for uncertainty quantification, strongly nonlinear systems through nonlinear normal modes, PDE-to-latent-ODE constructions via spectral or Galerkin reduction, and multi-physics coupling through block-diagonal physics matrices (Lai et al., 2022). NODE-ONet emphasizes flexibility across encoders, decoders, and related PDE families (&&&^^^^4\4^^^^5&&&). The aggregate direction of the field is therefore not toward a single standardized PENODE, but toward a broad class of hybrid continuous-time models in which mechanistic structure, numerical solvers, and trainable residuals are co-designed for the target scientific domain.