Accelerating operator Sinkhorn iteration with overrelaxation (2410.14104v1)

Abstract: We propose accelerated versions of the operator Sinkhorn iteration for operator scaling using successive overrelaxation. We analyze the local convergence rates of these accelerated methods via linearization, which allows us to determine the asymptotically optimal relaxation parameter based on Young's SOR theorem. Using the Hilbert metric on positive definite cones, we also obtain a global convergence result for a geodesic version of overrelaxation in a specific range of relaxation parameters. These techniques generalize corresponding results obtained for matrix scaling by Thibault et al. (Algorithms, 14(5):143, 2021) and Lehmann et al. (Optim. Lett., 16(8):2209--2220, 2022). Numerical experiments demonstrate that the proposed methods outperform the original operator Sinkhorn iteration in certain applications.

• The paper demonstrates that successive overrelaxation significantly accelerates the convergence of operator Sinkhorn iterations in operator scaling problems.
• It applies SOR techniques from numerical linear algebra to derive optimal relaxation parameters and establish both local and global convergence.
• Empirical evaluations confirm improved performance in balanced matrix transformations, suggesting broad applicability in computational mathematics.

Accelerating Operator Sinkhorn Iteration with Overrelaxation

The paper under consideration explores the acceleration of the operator Sinkhorn iteration for operator scaling through the use of successive overrelaxation (SOR). The primary focus is to achieve improved convergence rates by adapting techniques well-established in numerical linear algebra for accelerating matrix scaling problems to the more general scenario of operator scaling.

Operator Scaling and Its Applications

Operator scaling is an important computational problem where the objective is to transform a collection of matrices $A_1, \dots, A_k \in \mathbb{R}^{m \times n}$ such that the resulting transformations satisfy certain balance conditions. Formally, given matrices $L \in \text{GL}_m(\mathbb{R})$ and $R \in \text{GL}_n(\mathbb{R})$, the transformation should meet: $$\sum_{i=1}^k \bar{A}_i \bar{A}_i^\top = \frac{1}{m}I_m, \quad \sum_{i=1}^k \bar{A}_i^\top \bar{A}_i = \frac{1}{n}I_n$$ where $\bar{A}_i = L A_i R^\top$.

This problem has roots in finding applications in areas like non-commutative polynomial identity testing, invariant theory, and in computational statistics and signal processing.

Proposed Contributions

The authors propose accelerated variants of the operator Sinkhorn iteration using SOR, a method used to speed up iterative solutions of both linear and nonlinear systems. Local convergence rates of these methods are analyzed through linearization facilitated by Young's SOR theorem. Moreover, by utilizing the Hilbert metric on positive definite cones, the authors argue for global convergence for a geodesic version within specific relaxation parameter ranges.

Experimental Evaluation

Empirical results demonstrate that the proposed methods outperform the original operator Sinkhorn iteration in certain applications by achieving faster convergence rates. This is particularly evident in scenarios such as frame scaling applications.

Methods and Results

The paper systematically generalizes matrix scaling acceleration results to operator scaling. Highlights include:

• Variants of Overrelaxation: Different overrelaxation schemes are proposed, focusing on affine combinations, coordinate transforms, and geodesic paths on the space of positive definite matrices.
• Local Convergence Analysis: Each overrelaxation variant is analyzed for local convergence, with the derivation of an optimal relaxation parameter that minimizes spectral radius and thus maximizes convergence speed.
• Global Convergence: For the geodesic version, global convergence is established under conditions extending classical results of Sinkhorn and Birkhoff-Hopf for matrix scaling.

Implications and Future Directions

The results suggest that incorporating overrelaxation with geodesic considerations may offer substantial benefits in applications demanding efficient operator scaling. From a theoretical standpoint, the work extends the mathematical understanding of operator scaling in novel directions, exploring not just local but potential avenues for global convergence improvements.

Future developments could focus on further refining the methods for ill-conditioned cases and thoroughly exploring usage in diverse applications ranging from data sciences to quantum information processing.

In conclusion, the paper provides a well-grounded investigation into accelerating operator Sinkhorn iteration, contributing to both theoretical advancements and practical acceleration techniques in computational mathematics and data science applications.