Subspace curvature-scaling high-index saddle dynamics for accelerating ill-conditioned saddle point searches

Abstract: We propose a subspace curvature-scaling high-index saddle dynamics (SCS-HiSD) method to accelerate high-index saddle dynamics (HiSD) for locating ill-conditioned saddle points. The key observation is that HiSD already computes approximations of the unstable Hessian eigenvectors during iteration, which can be used at negligible additional cost to construct an inverse-Hessian approximation on the unstable subspace. This subspace curvature information is incorporated to adaptively scale the dynamics along each unstable direction, eliminating the dependence of the convergence rate on the smallest-magnitude negative eigenvalues and thereby substantially accelerating the convergence for ill-conditioned saddle points. We establish the linear stability of the continuous SCS-HiSD system and provide a local convergence analysis for the discrete iterative scheme. This method extends naturally to address slow convergence caused by small positive eigenvalues. Numerical experiments on benchmark problems and a liquid-crystal model demonstrate that SCS-HiSD substantially accelerates the computation of ill-conditioned saddle points, particularly in severely ill-conditioned cases.

• The paper demonstrates that SCS-HiSD leverages local curvature information to eliminate convergence bottlenecks in ill-conditioned saddle point searches.
• It repurposes eigenvector computations to form an inverse Hessian approximation, achieving linear convergence independent of small eigenvalues.
• Numerical experiments show up to a 900× speedup over traditional HiSD methods in both benchmark functions and liquid crystal energy landscapes.

Subspace Curvature-Scaling High-Index Saddle Dynamics for Accelerating Ill-Conditioned Saddle Point Searches

Motivation and Methodological Innovations

Locating saddle points of arbitrary index in complex high-dimensional energy landscapes is a fundamental problem in mathematical physics, chemistry, and materials science, underpinning the characterization of metastable transitions and bifurcations. High-index saddle dynamics (HiSD) methods have recently enabled systematic exploration of these landscapes, but their efficacy is severely impeded when targeting ill-conditioned saddle points—those whose Hessian matrices possess small-magnitude negative or positive eigenvalues. The principal convergence rate bottleneck arises from the slow evolution along nearly flat directions associated with these eigenvalues, making direct computation intractable for many practical scenarios.

This paper introduces the Subspace Curvature-Scaling High-Index Saddle Dynamics (SCS-HiSD) method, which leverages local curvature information of the energy landscape by adaptively scaling the search dynamics in the unstable subspace. The central insight is that HiSD iterations inherently construct approximate eigenvectors and eigenvalues of the Hessian along unstable directions, which can be repurposed to build an inverse-Hessian approximation on the unstable subspace with negligible additional computation. SCS-HiSD applies this operator to adaptively scale updates, eliminating the dependence of the convergence rate on the smallest-magnitude negative eigenvalues, and thus significantly accelerating saddle searches in ill-conditioned cases.

Theoretical Foundations: Stability and Convergence

The authors rigorously establish the linear stability of the continuous SCS-HiSD system, demonstrating that index-$k$ saddle points correspond to linearly stable equilibria of the SCS-HiSD dynamical system. Local convergence analysis of the explicit Euler discretization further shows that, unlike HiSD, the SCS-HiSD algorithm achieves linear convergence with a rate independent of the smallest-magnitude negative eigenvalues. The proof leverages contraction mapping arguments and explicit characterization of the impact of subspace projection errors. Both exact and inexact eigenvector cases are treated, and error bounds are derived in terms of the spectral gap and eigenvector approximation quality.

The method naturally extends to handle additional small-magnitude positive eigenvalues by including extra directions in the curvature-scaling scheme, thus mitigating the convergence slowdown on both sides of the saddle spectrum.

Numerical Experiments: Benchmarking and Robustness

Comprehensive numerical experiments validate the theoretical predictions and demonstrate the superior performance of SCS-HiSD against baseline HiSD and heavy-ball accelerated HiSD (A-HiSD) methods.

On benchmark energy functions such as the modified Strictly Convex 2 and Rosenbrock-type functions, SCS-HiSD accelerates convergence by multiple orders of magnitude, particularly as the ill-conditioning becomes severe. In moderately ill-conditioned cases, A-HiSD is competitive, but as the smallest negative eigenvalues approach zero, both HiSD and A-HiSD exhibit severe stagnation, while SCS-HiSD maintains robust linear convergence.

Figure 1: SCS-HiSD achieves much faster error decay than HiSD on a modified Strictly Convex 2 function, even with small negative eigenvalues.

Figure 2: In a modified Rosenbrock-type function, SCS-HiSD robustly overcomes stagnation due to increasingly ill-conditioned saddle points.

Application to Liquid Crystal Energy Landscapes

The methodology is further tested on nematic liquid crystal systems governed by the Landau–de Gennes free energy functional, which exhibits complex bifurcation structure and numerous saddle points. Here, HiSD and A-HiSD incur massive computational costs and stagnation due to the presence of nearly zero Hessian eigenvalues at critical points and bifurcations. SCS-HiSD (with additional directions included for small positive eigenvalues) enables both upward and downward saddle searches through bifurcation regions within a few thousand iterations. This translates into an empirical speedup of up to $900\times$ in wall time compared to HiSD.

Figure 3: Visualization of saddle states in nematic liquid crystal, used as targets in high-index saddle searches.

Figure 4: SCS-HiSD and its extensions dramatically reduce the gradient norm and converge rapidly in the liquid crystal saddle search, outperforming HiSD and A-HiSD.

Strong numerical results highlight that the iteration count in HiSD increases sharply as the smallest-magnitude Hessian eigenvalue approaches zero, while SCS-HiSD iteration counts remain unaffected, confirming the theoretical claim that SCS-HiSD eliminates the principal convergence bottleneck for ill-conditioned saddle points.

Implications and Future Directions

Practically, SCS-HiSD enables efficient exploration of complicated solution landscapes, with immediate applications to phase transitions, bifurcation analysis, stochastic escape rates, and data-driven energy landscape mapping. The negligible computational overhead and general compatibility with existing eigenvector solvers make the method applicable in both classical and machine learning contexts (e.g., neural network-based saddle dynamics (Liu et al., 2024)).

Theoretically, the results open avenues for further regularization via quasi-Newton approaches in both unstable and stable subspaces. While the current method accelerates along a fixed number of directions, generalizing the subspace scaling to cases with many small positive eigenvalues remains a challenging direction. Further development and rigorous analysis for these cases will be essential for systematic landscape enumeration and automated solution mapping in large-scale systems.

Conclusion

The SCS-HiSD method delivers a rigorous, computationally efficient solution for searching high-index saddle points in ill-conditioned energy landscapes by repurposing HiSD’s intrinsic eigenvector computations to perform adaptive curvature scaling. Theoretical analysis shows that the convergence rate is no longer bottlenecked by the smallest-magnitude negative (or positive) eigenvalues, and numerical experiments confirm robust acceleration in both benchmark and real physical systems. The approach is anticipated to become a standard tool in high-dimensional saddle searching and solution landscape mapping, with broad relevance across mathematical physics, data-driven modeling, and computational chemistry.