Determine whether two-dimensional RNN backpropagation causes ill-conditioned loss landscapes

Determine whether the combinatorial growth of backpropagation paths in the two-dimensional recurrent neural network architecture is the primary cause of the ill-conditioned loss landscape observed during natural-gradient optimization.

Background

The paper reports that minimum-step stochastic reconfiguration (minSR) substantially outperforms Adam for one-dimensional recurrent neural-network (RNN) wave functions but offers only comparable performance in two dimensions. The authors attribute this discrepancy to the severe ill-conditioning observed in the two-dimensional loss landscape and conjecture that the number of gradient-propagation paths grows combinatorially because each two-dimensional RNN cell receives hidden states from two neighboring directions.

Establishing whether this mechanism is actually responsible would clarify the source of the optimization instability and could guide the design of more stable two-dimensional architectures, including GridLSTM or LeakyLP cells, that might improve natural-gradient optimization without relying on increasingly sophisticated damping schemes.

References

We conjecture that the 2D RNN wave function architecture, inspired by the multi-dimensional RNN (MDRNN) introduced in Ref., may be the cause of the large discrepancy between the performance of natural gradients in 1D and 2D. In one dimension, each cell passes its hidden state to a single neighbor. In two dimensions, each cell receives hidden states from two neighbors. As a result, the number of paths along which gradients travel grows combinatorially with system size, leading to training instabilities.

— Quantum Geometric Tensor Preconditioning for Stable Training of Recurrent Neural Quantum States  (2608.18065 - Attar et al., 18 Aug 2026) in Section 3, Results subsection “Square Lattice $J_1-J_2$ Model”; Section 7, Conclusions