Convergence issues with complex-valued eigenbasis reparameterization in quasi-DEER

Determine the underlying numerical and algorithmic causes of the observed convergence difficulties when reparameterizing the state space with a complex-valued eigenbasis to approximately diagonalize the dynamics Jacobians in quasi-Newton parallel evaluation (quasi-DEER) of nonlinear state space models on GPUs, and develop numerically robust alternatives that retain the efficiency benefits of diagonalization while ensuring reliable convergence.

Background

To reduce the computational and memory costs of parallel Newton methods, the thesis proposes quasi-DEER, which replaces dense Jacobians with structured approximations such as diagonals. One strategy is to reparameterize the state so that the dynamics Jacobians are (approximately) diagonal in a new basis obtained from an eigen-decomposition.

In practice, the eigenbasis is often complex-valued. The authors report that this complex-valued reparameterization struggles with convergence—particularly on GPUs—suggesting numerical precision issues, but the concrete reasons are not fully understood. They note that using a real eigenbasis (e.g., via symmetrization) can be effective in some settings (such as Langevin dynamics in MCMC), but the general cause of the instability with complex-valued bases remains unresolved.