PG-VPINN Variants for Robust Neural PDEs

• PG-VPINNs are neural variational solvers that use DNN trial spaces with Petrov–Galerkin weak formulations to enhance stability, adaptivity, and error control.
• Key variants such as classical VPINN, RVPINN, CRVPINN, hybrid, and interface-penalty methods differ in test-function choices and loss constructions for robust performance.
• These approaches offer superior error reduction and convergence in challenging scenarios like singular perturbations, advection-dominated flows, and multi-material discontinuities.

Petrov–Galerkin Variational Physics-Informed Neural Networks (PG-VPINNs) are a family of neural variational solvers for partial differential equations (PDEs) in which deep neural networks serve as the nonlinear trial space and the residuals are enforced in a weak (variational) sense using separate, typically lower-dimensional, test spaces. This framework generalizes classical Physics-Informed Neural Networks (PINNs) by replacing point-collocation (strong form) loss with discrete Petrov–Galerkin weak residuals—often leading to improved accuracy, lower sensitivity to regularity, and enhanced robustness, especially in the context of singular perturbations, advection-dominated phenomena, rough coefficients, and multi-material discontinuities. Key PG-VPINN variants emphasize stability, adaptivity, and efficient error control through choices of test-function spaces, Gram-matrix normalization, and mixed strong–weak loss strategies.

1. Mathematical Formulation and Variational Principles

In the PG-VPINN paradigm, consider a generic linear PDE on a domain Ω: $$\mathcal{L}[u](x) = f(x),\quad x \in \Omega,\quad \mathrm{BC}[u](x) = g(x),\quad x \in \partial\Omega$$ The weak (variational) statement seeks $u \in V$ such that for all test functions $v \in W$,

$a(u, v) = \ell(v)$

where $a(u, v)$ encodes the bilinear form (e.g., $$a(u, v) = \int_\Omega \nabla u \cdot \nabla v \, dx$$ for Poisson) and $\ell(v)$ represents the linear functional associated with the right-hand side and boundary data.

Trial space: PG-VPINNs parameterize $u$ as a DNN $u_\theta(x) = \mathrm{NN}_\theta(x)$, with possible hard or soft enforcement of Dirichlet boundaries (e.g., multiplicative factors or loss penalties).

Test space: Chosen independently of the trial space; options include global orthogonal bases (Legendre polynomials, trigonometric functions), local supports (hat/basis splines), delta functions at collocation points, or indicator/test functions per element.

Residual structure: The loss is constructed from the variational residual,

$$R_j(\theta) = a(u_\theta, \varphi_j) - \ell(\varphi_j),\qquad j=1,\ldots,M$$

where $u \in V$0 is a basis of the test space.

Integration by parts is often employed to reduce the highest derivative order of $u \in V$1 consumed by the DNN, lowering training cost and improving stability (Kharazmi et al., 2019).

2. Key PG-VPINN Variants and Algorithmic Implementations

Multiple PG-VPINN variants have been formulated:

Classical VPINN (Petrov–Galerkin version)

• Uses a DNN trial space and a global (or elementwise) test space, enforcing residuals via quadrature.
• The loss is quadratic in the residual, summed over all test indices:

$u \in V$2

• Standard choices: trial = DNN, test = Legendre or sine basis for global, linear hats for local (Kharazmi et al., 2019).

Robust VPINN (RVPINN)

• Replaces the quadratic residual sum by the discrete dual norm of the residual:

$u \in V$3

which, via Riesz representation, becomes

$u \in V$4

where $u \in V$5 is the Gram matrix and $u \in V$6.

• Error estimator: Provable a posteriori bounds relating loss to the energy-norm error under inf-sup and Fortin-type assumptions (Rojas et al., 2023).

Collocation-based Robust VPINN (CRVPINN)

• Instantiates RVPINN with Dirac delta test functions at collocation points. The corresponding Gram matrix is a sparse Laplacian stencil.
• Achieves efficient training via sparse LU factorization of Gram, enabling O(N) batch updates without quadrature overhead. The loss remains equivalent (up to constants) to the H¹ norm error (Łoś et al., 2024).

Mixed Strong–Weak (Hybrid) PG-VPINN

• Combines strong-form residuals (as in PINN) and weak-form residuals (as in VPINN), with a tunable stabilization parameter $u \in V$7:

$u \in V$8

• Shown to enhance stability and control spurious oscillations in advection-dominated or singular perturbation regimes (Sikora et al., 2023).

Interface-Penalty PG-VPINN

• Extends the weak form with penalty terms at interior-element interfaces, analogous to discontinuous Galerkin (DG) interior-penalty stabilization.
• Enables robust capture of sharp boundary/interior layers without mesh refinement or specialized coordinate transforms (Kumar et al., 13 Sep 2025).

Tree-based Adaptive Integration PG-VPINN

• Employs spatially adaptive (quadtree/octree) integration cells for multi-phase and discontinuous materials.
• Reduces integration cost while maintaining bulk accuracy. Enables seamless integration of numerically generated boundary data via Galerkin Boundary Element Method (GBEM) (Rodopoulos et al., 29 Jun 2025).

3. Network Architectures, Loss Construction, and Training Protocols

Architectural considerations

• Standard: Fully connected feed-forward DNNs with 3–6 hidden layers and 20–60 neurons per layer, activation $u \in V$9 or ReLU (Sikora et al., 2023, Rojas et al., 2023, Kumar et al., 13 Sep 2025, Rodopoulos et al., 29 Jun 2025).
• Output structures: Scalar for 1D/2D PDEs, vector-valued for elasticity or Stokes.
• Hard enforcement of Dirichlet BCs (multiplicative ansatz) or strong-penalty loss terms.

Loss construction

• Weak-form quadrature: Global or local basis test functions, Gauss–Lobatto or elementwise Gaussian points.
• Strong-form/pointwise losses: Random or adaptive collocation points near boundary/interior layers.
• Gram matrix assembly & inversion for robust variants.
• Interface penalties for singularly perturbed or discontinuous problems.

Training and optimization

• Two-phase optimization: initial Adam (lr ≈ 10⁻³ to 1.25×10⁻³), refinement via L-BFGS.
• Training runs: 10⁴–10⁵ epochs, early stopping via stagnation.
• Collocation and test-space adaptivity: mesh and point placement, basis function order, quadrature refinement.

4. Benchmarks, Numerical Comparisons, and Error Estimates

Empirical results consistently show that PG-VPINN variants outperform standard PINN and even classic Galerkin and SUPG-inspired PINN in several regimes:

Method	MSE (1D adv-diff)	MSE (2D E-J)	Speed	Robust Layer Capture
Pure PINN	2.1×10⁻³	1.8×10⁻¹	500–1000 s	No (oscillatory)
Pure VPINN	1.5×10⁻³	1.1×10⁻¹	500–1000 s	No (under-resolved)
PG-VPINN (τ≃0.1)	4.2×10⁻⁴	6.0×10⁻²	200–640 s	Yes (stable/accurate)
SUPG-FEM	7.5×10⁻⁴	5.2×10⁻²	10–20 s	Yes

In 1D convection–diffusion with interior layers (ε = 10⁻³), PG-VPINN reduces max error by an order of magnitude over standard VPINN or PINN. In 2D heterogeneous elasticity, tree-based integration PG-VPINN achieves R² = 0.98–0.999 for inclusion and matrix displacement fields, outperforming strong-form PINN (R² ≲ 0.95) (Kumar et al., 13 Sep 2025, Rodopoulos et al., 29 Jun 2025).

Error estimator property: Robust variants (RVPINN, CRVPINN) provide energy-norm loss values tightly correlated with true discretization/approximation error: $v \in W$0 with known constants μ, α and explicit tracking during training (Rojas et al., 2023, Łoś et al., 2024).

5. Applications, Extensions, and Limitations

Domains of application

• Singularly perturbed BVPs, advection–diffusion/interior/boundary layer problems (Kumar et al., 13 Sep 2025, Sikora et al., 2023)
• Heterogeneous and multi-phase media (elasticity, architected solids) (Rodopoulos et al., 29 Jun 2025)
• Poisson, Stokes, and parabolic PDEs in 1D–2D (Rojas et al., 2023, Łoś et al., 2024)
• Structural mechanics with boundary element data, multiphase inclusions (Rodopoulos et al., 29 Jun 2025)

Notable extensions

• Tree/octree-based hp-adaptive integration
• Multi-material test-function selection (piecewise polynomials per domain)
• Penalty parameter learning for automatic interface stabilization
• Incorporation of semi-analytical/GBEM-generated auxiliary internal data

Limitations

• High-dimensional test spaces or fine quadrature grids demand commensurate computational cost
• Efficient Gram-matrix factorization/inversion is crucial in robust formulations; this may be challenging for large, unstructured domains (Łoś et al., 2024)
• Penalty parameters for interface stabilization require tuning (Kumar et al., 13 Sep 2025)
• Demonstrated mainly in moderate dimensions (1D, 2D), with partial extension to 3D (Rodopoulos et al., 29 Jun 2025)
• Extension to strongly non-linear or time-dependent systems requires problem-specific adaptation

6. Relationship to Classical and Other PINN-Based Approaches

Framework	Trial Space	Test Space	Loss Form	Stabilization/Adaptivity	Error Estimators
PINN	DNN	δ(x−xₖ)(points)	Strong residual	None/adaptive points	No
VPINN	DNN	Polys/hats	Quad. weak res.	None (Galerkin)	No
PG-VPINN	DNN	arbitrary (PG)	Weak+PG penalty	Yes (PG, penalties, adapt)	Yes (robust forms)
RVPINN	DNN	finite-dim	Dual norm (minres)	Yes (Gram, adapt)	Yes (provable)
CRVPINN	DNN	δ(x−xₖ) (grid)	Robust-collocation	Yes (sparse LU)	Yes (provable)

PG-VPINN generalizes and unifies the classical PINN (strong/collocation) and VPINN (weak/Galerkin) approaches by (i) enabling independent, often low-regularity test spaces, (ii) supporting hybrid losses that improve accuracy in challenging cases, and (iii) providing a geometric and algorithmic pathway to robust, mesh-free variational methods (Kharazmi et al., 2019, Sikora et al., 2023, Rojas et al., 2023, Łoś et al., 2024).

7. Future Directions and Open Challenges

Promising research avenues for PG-VPINN include:

• Extensions to fully non-linear systems, constrained optimization, and eigenvalue problems.
• Multi-physics and coupled PDEs (e.g., fluid–structure, thermoelastic).
• Data-driven/adaptive test space selection (basis learning).
• Fully meshless or irregular geometry support (extension of CRVPINN).
• Integration with boundary element kernels for high-fidelity simulation using solely boundary data.
• Automated parameter tuning and meta-learning of weak-form penalties and quadrature.
• GPU-optimized, scalable implementations for high-dimensional and real-time applications.

Continued development in these areas may further enhance the flexibility, generality, and efficiency of variational neural PDE solvers. The Petrov–Galerkin and robust-dual-norm principles underlying the most advanced PG-VPINN variants appear to furnish an optimal balance of mesh freedom, error control, and computational tractability (Rojas et al., 2023, Łoś et al., 2024, Rodopoulos et al., 29 Jun 2025).