A Dirac-Frenkel-Onsager principle: Instantaneous residual minimization with gauge momentum for nonlinear parametrizations of PDE solutions

Abstract: Dirac-Frenkel instantaneous residual minimization evolves nonlinear parametrizations of PDE solutions in time, but ill-conditioning can render the parameter dynamics non-unique. We interpret this non-uniqueness as a gauge freedom: nullspace directions that leave the time derivative unchanged can be used to select better-conditioned parameter velocities. Building on Onsager's minimum-dissipation principle, we introduce a history variable -- interpretable as momentum -- and inject it only along the nullspace directions. The resulting Dirac-Frenkel-Onsager dynamics preserve instantaneous residual minimization, in contrast to standard regularization that can introduce bias, while promoting temporally smooth parameter evolutions. Examples demonstrate that the approach leads to increased robustness in singular and near-singular regimes.

• The paper introduces the DFO principle, overcoming gauge ambiguity via Onsager momentum to achieve robust and unique parameter evolution.
• DFO injects momentum exclusively in nullspace directions, preserving Galerkin optimality and instantaneous residual minimization even in ill-conditioned regimes.
• Numerical experiments demonstrate that DFO reduces errors and improves temporal smoothness compared to traditional Dirac–Frenkel methods across various PDEs.

A Dirac-Frenkel-Onsager Principle for Robust Nonlinear Parametrizations of PDE Solutions

Introduction and Motivation

The Dirac-Frenkel (DF) principle underpins many local-in-time neural PDE solver schemes by enforcing instantaneous residual minimization through Galerkin-type projections. However, nonlinear trial manifolds—including neural network parametrizations—exhibit rank-deficient Jacobians, leading to non-uniqueness in parameter dynamics and severe ill-conditioning. This non-uniqueness manifests as gauge freedom: nullspace directions in parameter space leave the function-level time derivative invariant, creating ambiguity in parameter update selection. Classical remedies such as minimal-norm regularization (truncated SVD, Tikhonov) introduce bias and can render the dynamics qualitatively incorrect, particularly in regimes of tangent space collapse.

This paper introduces the Dirac-Frenkel-Onsager (DFO) principle, addressing DF's core failure modes in ill-conditioned or singular parameter regimes. DFO leverages Onsager's minimum-dissipation principle, injecting "momentum" only in nullspace directions, thus uniquely determining parameter evolution while preserving instantaneous residual minimization. Temporal smoothness is promoted, resulting in robust, reliable dynamics across singular and near-singular regimes.

Theoretical Formulation

Gauge Freedom and Non-Uniqueness

The DF principle projects the PDE residual onto the tangent space of the trial manifold, yielding a well-defined function-level update. Rank-deficient parameter Jacobians induce tangent space collapse—a situation where multiple parameter velocities correspond to the same function velocity. The normal equations reveal an affine family of admissible velocities: $\eta = \bar{\eta} + w$, $w \in \mathrm{null}(G)$, where $\bar{\eta}$ is chosen (typically minimal-norm), and $G$ is the Gram matrix formed from the Jacobian.

Onsager Momentum and Gauge Fixing

DFO introduces a history variable $m(t)$ evolving via an exponential moving average governed by Onsager's principle. This variable tracks past reference velocities, acting as a temporal low-pass filter. Importantly, gauge fixing uses $m(t)$ strictly in nullspace directions: the parameter velocity update becomes $\dot{\theta} = \bar{\eta} + \lambda P m$, where $P$ projects onto the nullspace, and $\lambda$ controls the injection strength. This preserves Galerkin-optimal dynamics in function space while smoothing parameter trajectories.

Numerical and Analytical Results

Robustness Against Tangent Space Collapse

DFO is analytically and numerically shown to escape the collapse scenarios where minimal-norm DF dynamics fail. For the wave collision problem, DF yields parameter updates orthogonal to critical separation directions, causing waves to lock together post-collision—a major qualitative failure. DFO, via momentum injection, restores motion along nullspace directions during collapse, enabling correct physical behavior.

(Figure 1)

Figure 1: Relative error over time for three representative PDEs, demonstrating that DFO consistently improves robustness and reduces error compared to minimal-norm DF approaches.

Error Suppression and Temporal Smoothness

DFO suppresses spurious background errors and erratic parameter velocities. In charged particle transport (Vlasov equation), DF injects dynamics-agnostic updates in nullspace directions, leading to high background error. DFO, by selecting coherent parameter velocities, localizes error support and maintains smooth evolution.

(Figure 2)

Figure 2: Pointwise error plots for charged particles; DFO (right) suppresses background error through informed nullspace velocity selection, compared with DF (left).

High-dimension Scalability

DFO scales efficiently to five-dimensional Fokker-Planck equations, representing stochastic particle evolution. Temporal smoothing and momentum injection yield accurate mean and covariance predictions, outperforming DF-based and randomized baseline methods in empirical $L^2$ errors.

Figure 3: DFO yields accurate mean and covariance trajectories for a 5D Fokker-Planck equation, avoiding erratic jumps observed in standard DF dynamics.

Restoration of Dynamics After Collapse

In advection-reaction problems, DF stalls after tangent-space collapse due to frozen parameter updates. DFO, via nullspace momentum, immediately restores evolution, yielding high accuracy through repeated collapse events.

Figure 4: Space-time plot of rotating detonation waves showing sharp features and wave propagation enabled by DFO.

Figure 5: Snapshots of solution dynamics for transport through a complex flow field, with DFO correctly capturing nonuniform transport.

Figure 6: Density evolution for charged particles in the Vlasov equation, with DFO capturing shearing and deformation.

Figure 7: Marginal density evolution for the fifth component in the Fokker-Planck solution, demonstrating DFO's temporal coherence.

Figure 8: Random projections of parameter velocities in the Vlasov experiment; DFO shows smooth, temporally coherent trajectories, unlike erratic DF updates.

Methodological Implications and Practical Impact

DFO provides a principled, computationally efficient mechanism to resolve parameter ambiguities inherent in nonlinear trial manifolds for PDEs. By restricting momentum injection to gauge directions, DFO avoids bias in function-level evolution and improves practical reliability in neural PDE solvers. Numerical experiments confirm negligible computational overhead versus standard DF, with significant gains in accuracy and robustness, especially in challenging high-dimensional and ill-conditioned regimes. Randomized variants further reduce computational cost for large-scale problems.

Theoretical Implications and Future Directions

The DFO principle reframes parameter non-uniqueness as gauge freedom, suggesting flexible templates for gauge fixing beyond Onsager-type momenta. The separation between instantaneous residual minimization and nullspace velocity selection maintains desirable variational properties without sacrificing robustness. DFO opens avenues for automated hyperparameter selection based on conditioning or residual metrics, and invites exploration of higher-order gauge fixing and history-free techniques. The explicit handling of gauge freedom may inform broader strategies for nonlinear parameter evolution in scientific ML and reduced modeling.

Conclusion

The Dirac-Frenkel-Onsager principle resolves parameter ambiguity in nonlinear PDE solvers by injecting temporal momentum exclusively along gauge (nullspace) directions, thereby preserving instantaneous residual minimization in function space. DFO improves robustness in singular and ill-conditioned regimes, suppresses background error, and yields temporally smooth parameter evolutions. Empirical evidence across a spectrum of PDEs attests to DFO's efficacy and scalability, with minimal computational overhead. DFO establishes a foundation for principled gauge fixing in time-evolving nonlinear parametrizations and suggests further innovations in neural PDE solving and scientific machine learning.