Physics-Pretrained VAE (P2VAE)

• P2VAE is a VAE framework that integrates known physical constraints into latent priors, decoder design, and regularization to enhance model interpretability.
• It employs techniques such as manifold priors, latent partitioning, and PDE-informed decoders to enforce physics compliance and improve extrapolation.
• Applications span nonlinear dynamics, inverse problems, and airfoil parameterization, consistently showing improved accuracy and robustness over traditional VAEs.

Searching arXiv for recent and foundational papers on physics-informed/integrated VAEs and related P2VAE usages. Physics-Pretrained Variational Autoencoder (P2VAE; Editor’s term) denotes a family of variational autoencoder constructions in which physical knowledge is embedded into the latent prior, latent geometry, latent dynamics, decoder, or auxiliary regularization so as to learn parsimonious, robust, and interpretable representations of physical systems. In the cited literature, the designation covers several closely related formulations rather than a single canonical architecture: manifold-latent VAEs for nonlinear dynamics of parameterized PDEs and mechanics, physics-integrated VAEs with grounded latent subspaces, FEM-regularized decoders for PDE inverse problems, Gaussian-process latent priors derived from Green’s functions, physics-aware airfoil generators, adversarially disentangled physics-informed VAEs, and mixture models over PDE-governed mechanistic subtypes (Lopez et al., 2020, Takeishi et al., 2021, Tait et al., 2020, Beckers et al., 2023, Pinnawala et al., 18 Sep 2025).

1. Conceptual scope and nomenclature

The central idea is to augment a data-driven generative model with physical structure that is known a priori. One formulation describes physics-integrated generative modeling as a class of hybrid or grey-box modeling in which the data-driven model is augmented with the physics knowledge governing the data distribution, with the intended consequences of controlled generation, improved extrapolation, and improved interpretability (Akhtar, 2024). Closely related work presents VAEs for learning nonlinear dynamics of physical systems by incorporating physical information into priors and by using general manifold latent space representations to encode geometric and topological structure (Lopez et al., 2020).

A second line of work grounds part of the latent space in an incomplete mechanistic model and leaves the remainder to trainable neural components. In this formulation, the objective is not merely to add a physics prior, but to preserve the semantics of the physics-based latent variables while still permitting learned corrections for missing physics (Takeishi et al., 2021). A third line embeds discretized PDE weak forms into the decoder as a regularizing term, thereby recasting Bayesian inverse problems into a VAE framework with physically structured decoding (Tait et al., 2020).

Domain-specific variants extend the same general principle. In airfoil parameterization, physics awareness is expressed through a thickness/camber decomposition, B-spline decoding, and direct alignment of selected latent dimensions with geometric features (Kang et al., 2023). In structural-health and signal settings, adversarial disentanglement is used to keep known physics in one latent subspace and confounding influences in others (Koune et al., 16 Jun 2025). In neurodegeneration modeling, the framework expands to mixtures of PDE-governed latent dynamic models, explicitly departing from the assumption that a single PDE structure suffices for the whole population (Pinnawala et al., 18 Sep 2025).

2. Latent structure and decoder design

A recurrent architectural pattern is the introduction of a latent variable with explicit physical meaning. For nonlinear dynamics learned from state pairs $X=u(t)\in\mathbb{R}^d$ and $x=u(t+\tau)\in\mathbb{R}^d$, one formulation places $z\in\mathcal{M}$ on a prescribed manifold of dimension $m\ll d$, uses a prior $p(z)$ encoding physics knowledge, a decoder $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$, and an encoder $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$, where $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$ projects onto the manifold (Lopez et al., 2020). This design is explicitly topological: $S^1$, $S^1\times S^1$, and even a Klein bottle are treated as admissible latent spaces when dictated by the physical state.

Another recurrent pattern is latent partitioning. In the physics-integrated VAE of Takeishi and Kalousis, the latent space is split into a “physics” subspace $x=u(t+\tau)\in\mathbb{R}^d$0 and a “data-driven” subspace $x=u(t+\tau)\in\mathbb{R}^d$1, with decoder $x=u(t+\tau)\in\mathbb{R}^d$2 combining a mechanistic model $x=u(t+\tau)\in\mathbb{R}^d$3 and a learned complement $x=u(t+\tau)\in\mathbb{R}^d$4 through a general functional $x=u(t+\tau)\in\mathbb{R}^d$5 (Takeishi et al., 2021). A closely related factorization appears in the attentive planar normalizing-flow model, which posits

$x=u(t+\tau)\in\mathbb{R}^d$6

with $x=u(t+\tau)\in\mathbb{R}^d$7 implemented by an ODE solver and $x=u(t+\tau)\in\mathbb{R}^d$8 implemented by an MLP (Akhtar, 2024).

More specialized variants expand this partitioning further. The adversarially disentangled architecture uses $x=u(t+\tau)\in\mathbb{R}^d$9 for physics-grounded variables, $z\in\mathcal{M}$0 for domain factors, and $z\in\mathcal{M}$1 for class factors, with a decoder that sums a physics-based forward model $z\in\mathcal{M}$2 and a corrective neural branch $z\in\mathcal{M}$3 downstream of a Gradient Reversal Layer (Koune et al., 16 Jun 2025). The neurodegeneration mixture model introduces a categorical latent variable $z\in\mathcal{M}$4 that selects among distinct PDE subtypes, each with its own reaction form and rates $z\in\mathcal{M}$5 (Pinnawala et al., 18 Sep 2025).

Decoder design is correspondingly heterogeneous. Some models decode through analytic or differentiable physics solvers, including ODE solvers, finite-element solvers, and PDE simulators (Akhtar, 2024, Koune et al., 16 Jun 2025, Pinnawala et al., 18 Sep 2025). Others use fixed geometric layers: the airfoil generator decodes latent variables into B-spline control points for thickness and camber distributions, then reconstructs the airfoil through a fixed cubic B-spline layer evaluated at 101 chordwise points (Kang et al., 2023). A different route places the physical structure in the prior rather than the decoder: the physics-enhanced GP-VAE uses a Gaussian-process prior over latent trajectories whose covariance is induced by a known linear dynamical system through the corresponding Green’s function (Beckers et al., 2023).

3. Objective functions and the problem of enforcing physics usage

All variants remain recognizably variational, but the standard ELBO is almost always modified. In the manifold-latent dynamics model, training maximizes a minibatch ELBO

$z\in\mathcal{M}$6

where $z\in\mathcal{M}$7 is reconstruction, the KL term regularizes against a physics-informed prior, and $z\in\mathcal{M}$8 is a reconstruction-regularization term promoting multi-step consistency through a learned latent-dynamics map $z\in\mathcal{M}$9 (Lopez et al., 2020). This explicit $m\ll d$0-weighted term is central to the reported multi-step behavior.

In the physics-integrated VAE of Takeishi and Kalousis, the ELBO is supplemented by two regularizers designed to prevent the trainable component from overwhelming the mechanistic one: a posterior predictive-check regularizer $m\ll d$1 and physics-based data-augmentation terms $m\ll d$2 and $m\ll d$3. The overall objective is

$m\ll d$4

reflecting the stated technical challenge of striking a balance between incomplete physics and trainable components (Takeishi et al., 2021).

The attentive planar normalizing-flow variant introduces a factorized NF encoder and augments the ELBO with planar-flow log-determinant corrections and two Takeshi regularizers $m\ll d$5, yielding

$m\ll d$6

This model also applies scaled-dot-product self-attention to the latent variables after the flow layers and merges the attention outputs multiplicatively and additively, $m\ll d$7 and $m\ll d$8, to mitigate encoder noise and outliers (Akhtar, 2024).

Other papers encode physics as an explicit residual penalty. In variational autoencoding of PDE inverse problems, the relaxed constrained variational objective contains a term

$m\ll d$9

with a stochastic mini-patch approximation used to avoid cubic-cost inversions (Tait et al., 2020). In the neurodegeneration mixture model, the loss is negative ELBO plus a PDE residual regularizer weighted by $p(z)$0, ensuring decoded trajectories satisfy the selected reaction-diffusion equation (Pinnawala et al., 18 Sep 2025). In the adversarially disentangled model, the Gradient Reversal Layer induces a minimax effect that maximizes the reconstruction contribution with respect to the encoder parameters associated with the data-driven factors while leaving the auxiliary and KL terms as minimization, thereby discouraging $p(z)$1 and $p(z)$2 from encoding purely physical variation (Koune et al., 16 Jun 2025). This suggests that, within the P2VAE family, “physics-informed” is as much a statement about training constraints as about architecture.

4. Forms of physical prior knowledge

One important mode of integration is geometric and topological prior knowledge. When the state naturally lies on a manifold, the latent prior and the encoder projection are chosen to respect that manifold. For $p(z)$3 and $p(z)$4, analytic projections are available; for more general manifolds, nearest-point projection onto a high-resolution point cloud with local chart-based gradients is used. The latent prior can be uniform or wrapped Gaussian on $p(z)$5, thereby encoding topological constraints and symmetries rather than only scale information (Lopez et al., 2020).

A second mode is decoder-side mechanistic knowledge. Physics-grounded decoders range from simple analytic forms to numerical solvers. Reported examples include a pendulum model in which $p(z)$6 approximates natural frequency, an advection-diffusion model in which $p(z)$7 approximates a diffusion coefficient, Euler-Bernoulli beam equations for displacement fields, a damped oscillator with $p(z)$8, finite-element bridge dynamics, and reaction-diffusion PDE solvers for neuroimaging trajectories (Takeishi et al., 2021, Koune et al., 16 Jun 2025, Pinnawala et al., 18 Sep 2025). In inverse problems, FEM assembly is inserted directly into the decoder path to compute physics residuals from a surrogate solution $p(z)$9 (Tait et al., 2020).

A third mode is physics-aware parametrization of the output space. The airfoil generator does not rely on CFD labels; instead it explicitly separates thickness $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$0 and camber $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$1, predicts B-spline control points for both, enforces non-decreasing $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$2 coordinates by predicting nonnegative $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$3, and enforces $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$4 by constraining the thickness control-point ordinates. Four latent dimensions are reserved for normalized maximum camber, normalized trailing-edge angle, normalized maximum thickness, and normalized leading-edge radius, with an MSE-based physics loss applied both on decoded training samples and on random latent samples decoded within the bounding box of the training latents (Kang et al., 2023).

A fourth mode is physics-structured stochastic process priors. In the physics-enhanced GP-VAE, the prior $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$5 is not a generic temporal prior but a GP covariance induced by a linear dynamical system $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$6, $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$7, with the kernel derived from the corresponding Green’s function. The physics thereby enters the KL term through the covariance matrix $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$8 (Beckers et al., 2023).

Finally, the mixture model introduces heterogeneous mechanistic subtypes. Rather than requiring all subjects to follow a single PDE structure, it posits $$p_\theta(x\mid z)\approx b_\theta(z)+\mathcal{N}(0,\sigma_d^2 I)$$9 PDE components, each with its own reaction form $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$0 and physical parameters $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$1, and infers subtype membership by a categorical latent variable (Pinnawala et al., 18 Sep 2025). A plausible implication is that this formulation treats model misspecification itself as a latent structure-learning problem.

5. Representative applications and quantitative findings

Setting	Reported result	Source
Nonlinear Burgers’ equation	P2VAE with $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$2 achieves $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$3-relative error $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$4 up to $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$5; linear VAE $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$6, DMD (3 modes) $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$7, POD (3 modes) $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$8	(Lopez et al., 2020)
Constrained mechanical arm	Reconstruction $$q_\phi(z\mid X)\approx \Lambda(\tilde a_\phi(X))+\mathcal{N}(0,\sigma_e^2 I)$$9 errors after $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$0 epochs: P2VAE on $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$1 $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$2; VAE on $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$3 $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$4; VAE on $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$5 $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$6	(Lopez et al., 2020)
Human locomotion	MAE: Ord-VAE $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$7, Phy-VAE $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$8, Ord+Phy+R-VAE $\Lambda:\mathbb{R}^{2m}\to\mathcal{M}$9, NF+Phy+R-VAE $S^1$0, Att-NF+Phy+R-VAE $S^1$1	(Akhtar, 2024)
Airfoil parameterization	Feasibility under non-intersection: AG$S^1$2, AG$S^1$3 $S^1$4; max Pearson correlation for AG $S^1$5 over $S^1$6	(Kang et al., 2023)
Pendulum and advection–diffusion	$S^1$7-inference MAE $S^1$8; $S^1$9-inference MAE $S^1\times S^1$0 when moving from NN+phys to NN+phys+reg	(Takeishi et al., 2021)
Aquifer groundwater-level data	MAE $S^1\times S^1$1 for GP $S^1\times S^1$2 GP-D $S^1\times S^1$3 GP-DT	(Tait et al., 2020)
Synthetic PDE-mixture benchmark	Inferred subtype accuracy $S^1\times S^1$4; extrapolation MSE $S^1\times S^1$5	(Pinnawala et al., 18 Sep 2025)

These results are heterogeneous in task, metric, and data modality, but several recurrent empirical patterns are explicit in the literature. In the Burgers study, $S^1\times S^1$6 is reported as key to organizing concentric latent circles and stable extrapolation, and removing $S^1\times S^1$7 degrades multi-step prediction and scatters latent trajectories (Lopez et al., 2020). In the constrained mechanics setting, the wrong topology forces the encoder to “double back” and greatly increases reconstruction loss, even without noise, while the manifold-constrained model remains far more robust under input noise $S^1\times S^1$8 up to $S^1\times S^1$9 (Lopez et al., 2020).

The human-locomotion study shows that planar flows can improve reconstruction fidelity over both ordinary VAE and prior physics-integrated baselines, while the attention-augmented model exhibits an unusual MAE decrease up to approximately $x=u(t+\tau)\in\mathbb{R}^d$00 feature corruption, attributed to the contextual-attention mechanism (Akhtar, 2024). The airfoil study reports that explicit feasibility constraints can eliminate self-intersecting samples entirely and that reserved physical latent dimensions can achieve nearly one-to-one correspondence with selected geometric features (Kang et al., 2023). The Takeishi–Kalousis results further indicate that reconstruction quality alone is not the decisive criterion: the regularized model is the one that materially improves inference of the physics parameter and preserves extrapolative behavior beyond the training interval (Takeishi et al., 2021).

6. Misconceptions, limitations, and open directions

A frequent misconception is that P2VAE refers to a single fixed architecture. The literature instead spans manifold priors, latent partitioning, PDE-regularized decoders, Gaussian-process latent dynamics, explicit geometry-aware decoders, adversarial disentanglement, and mixture models over PDE families (Lopez et al., 2020, Takeishi et al., 2021, Beckers et al., 2023, Pinnawala et al., 18 Sep 2025). The common element is the attempt to constrain generative modeling by physical structure; the implementation details differ substantially across domains.

A second misconception is that the mere presence of a physics module guarantees interpretability. Several papers state the opposite problem explicitly: a trainable neural branch can “swallow” the signal, or the physics and neural components can share physical variation in a way that destroys semantic separation. This is why the literature introduces posterior predictive-check and data-augmentation regularizers, Takeshi regularizers, gradient reversal, conditional priors, or PDE residual penalties to ensure that the physics component is actually used in a meaningful manner (Takeishi et al., 2021, Akhtar, 2024, Koune et al., 16 Jun 2025). The adversarially disentangled beam example makes the point sharply: without adversarial control, $x=u(t+\tau)\in\mathbb{R}^d$01 overfits the physical parameter $x=u(t+\tau)\in\mathbb{R}^d$02, whereas with $x=u(t+\tau)\in\mathbb{R}^d$03, $x=u(t+\tau)\in\mathbb{R}^d$04 and $x=u(t+\tau)\in\mathbb{R}^d$05 become invariant to $x=u(t+\tau)\in\mathbb{R}^d$06 and the $x=u(t+\tau)\in\mathbb{R}^d$07-variation moves to $x=u(t+\tau)\in\mathbb{R}^d$08 (Koune et al., 16 Jun 2025).

A third misconception concerns the word “pretrained.” In several implementations, no separate physics-based pretraining stage is used; the model is trained end-to-end from scratch (Takeishi et al., 2021, Kang et al., 2023). In other cases, pretraining is optional rather than mandatory: when $x=u(t+\tau)\in\mathbb{R}^d$09 is not analytic, one may pretrain a small neural-net surrogate on simulator data so that the VAE does not have to learn the nominal forward model from limited and noisy observations alone (Koune et al., 16 Jun 2025). The term therefore should not be read as implying a uniform training protocol across the entire family.

Open problems are described repeatedly and with some consistency. For manifold-latent dynamics, stated challenges include explicit PDE constraints in the decoder “a la PINNs,” jointly learning the manifold geometry instead of prescribing $x=u(t+\tau)\in\mathbb{R}^d$10, scalability to very high-dimensional fields such as 2D/3D fluids using convolutional encoder/decoder architectures, guarantees of long-horizon stability, and quantifying uncertainty propagation through the physics-informed prior (Lopez et al., 2020). For FEM-regularized inverse problems, the unresolved issues include computational overhead, sensitivity to the $x=u(t+\tau)\in\mathbb{R}^d$11-schedule and taper radius, and extension beyond the demonstrated elliptic or steady PDE settings (Tait et al., 2020). For physics-enhanced GP priors, scalability of full GP covariances to long sequences and extension from linear to nonlinear or time-varying physical models remain open (Beckers et al., 2023). For PDE-based disease modeling, a major motivation is precisely that current physics-integrated machine learning methods are limited to considering a single PDE; the mixture model addresses this by learning multiple mechanistic subtypes, but higher-resolution 3D imaging and richer priors remain natural extensions (Pinnawala et al., 18 Sep 2025).

Taken together, the P2VAE literature defines a research program rather than a single model class: build a VAE whose latent variables, priors, decoder, or optimization objective are constrained by geometry, topology, mechanistic equations, or scientifically meaningful features, and then use those constraints to obtain reduced models that are more robust, more interpretable, and more physically compliant than unconstrained black-box counterparts (Lopez et al., 2020, Takeishi et al., 2021, Akhtar, 2024).