When PINNs Go Wrong: Pseudo-Time Stepping Against Spurious Solutions

Abstract: Physics-informed neural networks (PINNs) provide a promising machine learning framework for solving partial differential equations, but their training often breaks down on challenging problems, sometimes converging to physically incorrect solutions despite achieving small residual losses. This failure, we argue, is not merely an optimization difficulty. Rather, it reflects a fundamental weakness of the empirical PDE residual loss, which can admit trivial or spurious solutions during training. From this perspective, we revisit pseudo-time stepping, a technique that has recently shown strong empirical success in PINNs. We show that its main benefit is not simply to ease optimization; instead, when combined with collocation-point resampling, it helps reveal and avoid spurious solutions. At the same time, we find that the effectiveness of pseudo-time stepping depends critically on the choice of step size, which cannot be tuned reliably from the training loss alone. To overcome this limitation, we propose an adaptive pseudo-time stepping strategy that selects the step size from a finite-difference surrogate of the local residual Jacobian, yielding the largest step permitted by local stability without per-problem tuning. Across a diverse set of PDE benchmarks, the proposed method consistently improves both accuracy and robustness. Together, these findings provide a clearer understanding of why PINNs fail and suggest a practical pathway toward more reliable physics-informed learning. All code and data accompanying this manuscript are available at https://github.com/sifanexisted/jaxpi2.

• The paper establishes that finite collocation in PINNs allows spurious solutions, demonstrated through analytical mollifier techniques.
• The study reveals that pseudo-time stepping destabilizes nonphysical solutions by amplifying localized residual differences.
• The research introduces an adaptive pseudo-time stepping algorithm that reduces L2 errors by 1-2 orders on challenging PDE benchmarks.

Pseudo-Time Stepping for Mitigating Spurious Solutions in Physics-Informed Neural Networks

Introduction

This work provides a significant theoretical and empirical dissection of failure modes in physics-informed neural networks (PINNs), specifically focusing on the tendency to converge to spurious or trivial solutions when solving PDEs. The authors attribute this failure not solely to optimization difficulties (such as ill-conditioning or gradient pathologies), but to a fundamental ill-posedness associated with the empirical loss: the mean-squared residual is enforced only at a finite collocation set, permitting the existence of nonphysical solutions that interpolate all collocation constraints while failing elsewhere.

Importantly, the paper analyzes and formalizes how pseudo-time stepping—a variant of pseudo-transient continuation integrated into the PINN loss—can suppress convergence to such spurious solutions. The authors identify both the benefits and limitations of pseudo-time stepping and introduce an adaptive strategy for choosing its step size automatically.

Analysis of Spurious Solution Formation in PINNs

The central theoretical contribution is a formal demonstration (Theorem 2.1) that, for any finite set of collocation points and homogeneous PDE data, there exist smooth but spurious solutions that perfectly interpolate all residual constraints yet are nonphysical globally. The core idea is to use mollifier techniques to construct functions that match the true solution only at the collocation points and default to trivial behavior (such as zero) elsewhere. This exposes a crucial limitation: empirical residual minimization at finitely many sites does not guarantee global PDE satisfaction.

The prevalence of this ill-posedness is empirically illustrated via canonical forward and boundary-value PDEs, such as 1D advection and 2D lid-driven cavity flows. In both cases, PINNs trained by minimizing the empirical loss saturate at nonphysical states, with L2 errors exceeding 80–90%. This phenomenon is robust across network architectures, weighting, and optimizers. Additionally, the authors clarify that pathologies such as spectral bias, loss-term gradient conflicts, or optimizer-induced issues alone cannot resolve or explain these spurious solutions.

Mechanistic Role of Pseudo-Time Stepping

Contrary to conventional wisdom attributing the efficacy of pseudo-time stepping to improved optimization landscapes, the authors demonstrate that its main role is in revealing and destabilizing spurious solutions via residual amplification, especially under collocation-point resampling.

The pseudo-time stepping modification incorporates an auxiliary artificial time and a step-size parameter $\tau$, replacing the standard PDE residual with a relaxed residual between the current and previous iterates and the target PDE operator. Theoretical analysis (Theorem 2.5) shows that pseudo-time stepping, when combined with collocation resampling, amplifies the hidden defects of spurious solutions by raising the residual in localized transition regions. Thus, it acts not through smoother optimization convergence, but by ensuring that previously undetectable nonphysical interpolants are more likely to be penalized and avoided in parameter updates.

Critically, the effectiveness of pseudo-time stepping depends on the choice of $\tau$. Too large a $\tau$ can destabilize optimization, whereas too small values limit the amplification effect.

Adaptive Pseudo-Time Step Size Selection

The key practical limitation identified is the sensitivity of PINN training to the pseudo-time step $\tau$. The optimal value is problem dependent and cannot be reliably selected using the training loss, since spurious solution minima often exhibit small empirical losses. Manual $\tau$ tuning is thus not feasible for practical deployments.

To alleviate this, the authors design an adaptive pseudo-time stepping algorithm that updates $\tau$ dynamically based on finite-difference estimates of the local Jacobian of the PDE residual operator (in the function space, not the parameter space). Specifically, the update is governed by the ratio of the norm of the output change to the norm of the residual change across consecutive iterates, modulated by a cosine decay schedule to shrink $\tau$ as training converges. This Barzilai–Borwein-type estimator ensures that the step-size tracks the local stiffness of the problem without costly Jacobian computations.

Componentwise adaptive step sizes are also introduced for systems of PDEs, providing further flexibility for multi-equation settings.

Empirical Results

Comprehensive experiments on ten challenging PDE benchmarks—encompassing classical equations exhibiting shocks, chaotic dynamics, and multiscale patterns (e.g., inviscid Burgers, Kuramoto-Sivashinsky, Gray-Scott, high-Re incompressible flow)—demonstrate the robust gains of adaptive pseudo-time stepping over both fixed-$\tau$ and state-of-the-art baseline methods. Relative $L^2$ errors are consistently and significantly reduced, particularly on problems prone to spurious solution convergence (errors drop by one to two orders of magnitude compared to the strong baseline).

Ablation studies indicate robustness to architecture/optimizer selection and loss weighting strategies: pseudo-time stepping yields gains regardless of backbone (PirateNet vs. MLP), optimizer (SOAP vs. Adam), and with or without causal temporal weighting. The adaptive update frequency for $\tau$ is critical; both overly frequent and overly infrequent adaptation can degrade solution quality, with per-1000-iteration updates providing an optimal trade-off.

Computational overhead incurred by pseudo-time stepping is modest, since it primarily involves an auxiliary forward pass of the network without additional backpropagation.

Implications and Future Directions

The work provides a rigorous perspective for understanding when and why PINNs fail to find physically meaningful solutions and when they converge to nonphysical interpolants. The identification of the ill-posedness of the empirical residual loss and the mechanism of spurious solution destabilization via pseudo-time stepping is theoretically significant. This perspective clarifies that loss value and optimizer performance alone are not adequate metrics for solution fidelity; direct strategies for residual amplification and detection of nonphysical modes are essential.

The adaptive pseudo-time stepping method is architecturally and algorithmically lightweight, generalizes to a wide range of problems, and is compatible with other recent PINN advances (architectural, weighting, and optimization). The methodology can be directly extended to other classes of physics-informed models, such as neural operators and operator-learning architectures, as well as to more complex multiphysics and domain-decomposition settings.

Potential further improvements include developing adaptive strategies based on longer history, spatially/temporally localized step-size adaptation, or more sophisticated surrogates for the local residual Jacobian. Application areas beyond the fluid-testing domain, especially in computational solid mechanics and challenging multi-physics inverse problems, represent promising directions.

Conclusion

This study elucidates a foundational failure mode in PINNs arising from the empirical residual formulation, rigorously characterizes the existence and impact of spurious solutions, and introduces pseudo-time stepping—not as an ad-hoc optimizer acceleration, but as a principled mechanism for solution disambiguation. The proposed adaptive pseudo-time stepping strategy yields robust accuracy and eliminates the need for problem-specific hyperparameter tuning. These results represent a substantial advance in the scalable, reliable deployment of PINNs for high-dimensional, nonlinear PDEs (2604.23528).