Narrowing the computational-efficiency gap of Riemannian TTN optimizers

Determine whether the computational-efficiency gap between Riemannian optimizers for orthogonal tree tensor networks and unconstrained optimization can be narrowed sufficiently that the additional cost of Riemannian optimization becomes negligible.

Background

The paper develops stochastic Riemannian optimizers for orthogonal tree tensor networks, including RADAM, RDOG, and RMUON, and compares them with unconstrained ADAM. Although the Riemannian methods provide numerically stable iterates suitable for downstream operations such as compression, their projections and retractions introduce computational overhead, and the experiments show that unconstrained ADAM is generally faster.

The authors explicitly leave unresolved whether further algorithmic improvements can reduce this overhead to a negligible level while retaining the geometric and numerical advantages of Riemannian optimization.

References

In general, there is still room for improvement in the computational efficiency of the Riemannian optimizers, and it remains to be seen whether the gap to unconstrained optimization can be further narrowed so that the additional cost becomes negligible.

— Stochastic Optimization of Tree Tensor Networks  (2609.00870 - Willner et al., 1 Sep 2026) in Section 6, Conclusion