Local Geometry Determines Global Landscape in Low-rank Factorization for Synchronization (2311.18670v1)
Abstract: The orthogonal group synchronization problem, which focuses on recovering orthogonal group elements from their corrupted pairwise measurements, encompasses examples such as high-dimensional Kuramoto model on general signed networks, $\mathbb{Z}_2$-synchronization, community detection under stochastic block models, and orthogonal Procrustes problem. The semidefinite relaxation (SDR) has proven its power in solving this problem; however, its expensive computational costs impede its widespread practical applications. We consider the Burer-Monteiro factorization approach to the orthogonal group synchronization, an effective and scalable low-rank factorization to solve large scale SDPs. Despite the significant empirical successes of this factorization approach, it is still a challenging task to understand when the nonconvex optimization landscape is benign, i.e., the optimization landscape possesses only one local minimizer, which is also global. In this work, we demonstrate that if the degree of freedom within the factorization exceeds twice the condition number of the Laplacian" (certificate matrix) at the global minimizer, the optimization landscape is absent of spurious local minima. Our main theorem is purely algebraic and versatile, and it seamlessly applies to all the aforementioned examples: the nonconvex landscape remains benign under almost identical condition that enables the success of the SDR. Additionally, we illustrate that the Burer-Monteiro factorization is robust to
monotone adversaries", mirroring the resilience of the SDR. In other words, introducing ``favorable" adversaries into the data will not result in the emergence of new spurious local minimizers.
- E. Abbe. Community detection and stochastic block models: recent developments. The Journal of Machine Learning Research, 18(1):6446–6531, 2017.
- Decoding binary node labels from censored edge measurements: Phase transition and efficient recovery. IEEE Transactions on Network Science and Engineering, 1(1):10–22, 2014.
- Exact recovery in the stochastic block model. IEEE Transactions on information theory, 62(1):471–487, 2015.
- Entrywise eigenvector analysis of random matrices with low expected rank. Annals of Statistics, 48(3):1452, 2020.
- Expander graphs are globally synchronising. arXiv preprint arXiv:2210.12788, 2022.
- Optimization algorithms on matrix manifolds. Princeton University Press, 2008.
- A. S. Bandeira. Random Laplacian matrices and convex relaxations. Foundations of Computational Mathematics, 18(2):345–379, Apr 2018.
- Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. Mathematical Programming, 163(1-2):145–167, 2017.
- On the low-rank approach for semidefinite programs arising in synchronization and community detection. In Conference on Learning Theory, pages 361–382, 2016.
- Open problem: Tightness of maximum likelihood semidefinite relaxations. In Conference on Learning Theory, pages 1265–1267. PMLR, 2014.
- A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications. SIAM, 2001.
- N. Boumal. A riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints. arXiv preprint arXiv:1506.00575, 2015.
- N. Boumal. Nonconvex phase synchronization. SIAM Journal on Optimization, 26(4):2355–2377, 2016.
- N. Boumal. An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023.
- Deterministic guarantees for burer-monteiro factorizations of smooth semidefinite programs. Communications on Pure and Applied Mathematics, 73(3):581–608, 2020.
- S. Burer and R. D. Monteiro. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Mathematical Programming, 95(2):329–357, 2003.
- S. Burer and R. D. Monteiro. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103(3):427–444, 2005.
- Global registration of multiple point clouds using semidefinite programming. SIAM Journal on Optimization, 25(1):468–501, 2015.
- Y. Cheng and R. Ge. Non-convex matrix completion against a semi-random adversary. In Conference On Learning Theory, pages 1362–1394. PMLR, 2018.
- U. Feige and J. Kilian. Heuristics for semirandom graph problems. Journal of Computer and System Sciences, 63(4):639–671, 2001.
- Matrix completion has no spurious local minimum. Advances in neural information processing systems, 29, 2016.
- Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6):1115–1145, 1995.
- J. C. Gower. Generalized Procrustes analysis. Psychometrika, 40(1):33–51, 1975.
- Achieving exact cluster recovery threshold via semidefinite programming: Extensions. IEEE Transactions on Information Theory, 62(10):5918–5937, 2016.
- Q.-X. Huang and L. Guibas. Consistent shape maps via semidefinite programming. Computer Graphics Forum, 32(5):177–186, 2013.
- Sufficiently dense Kuramoto networks are globally synchronizing. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(7), 2021.
- A global synchronization theorem for oscillators on a random graph. Chaos: An Interdisciplinary Journal of Nonlinear Science, 32(9), 2022.
- Y. Kuramoto. Self-entrainment of a population of coupled non-linear oscillators. In International Symposium on Mathematical Problems in Theoretical Physics: January 23–29, 1975, Kyoto University, Kyoto/Japan, pages 420–422. Springer, 1975.
- S. Ling. Improved performance guarantees for orthogonal group synchronization via generalized power method. SIAM Journal on Optimization, 32(2):1018–1048, 2022.
- S. Ling. Near-optimal bounds for generalized orthogonal procrustes problem via generalized power method. Applied and Computational Harmonic Analysis, 66:62–100, 2023.
- S. Ling. Solving orthogonal group synchronization via convex and low-rank optimization: Tightness and landscape analysis. Mathematical Programming, 200(1):589–628, 2023.
- On the landscape of synchronization networks: A perspective from nonconvex optimization. SIAM Journal on Optimization, 29(3):1879–1907, 2019.
- On the estimation performance and convergence rate of the generalized power method for phase synchronization. SIAM Journal on Optimization, 27(4):2426–2446, 2017.
- A unified approach to synchronization problems over subgroups of the orthogonal group. Applied and Computational Harmonic Analysis, 66:320–372, 2023.
- Almost global consensus on the n𝑛nitalic_n-sphere. IEEE Transactions on Automatic Control, 63(6):1664–1675, 2017.
- High-dimensional Kuramoto models on Stiefel manifolds synchronize complex networks almost globally. Automatica, 113:108736, 2020.
- Point registration via efficient convex relaxation. ACM Transactions on Graphics (TOG), 35(4):1–12, 2016.
- A. D. McRae and N. Boumal. Benign landscapes of low-dimensional relaxations for orthogonal synchronization on general graphs. arXiv preprint arXiv:2307.02941, 2023.
- Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality. In Conference on Learning Theory, pages 1476–1515, 2017.
- Registration of point cloud data from a geometric optimization perspective. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pages 22–31, 2004.
- How robust are reconstruction thresholds for community detection? In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, pages 828–841, 2016.
- Y. Nesterov and A. Nemirovskii. Interior-Point Polynomial Algorithms in Convex Programming. SIAM, 1994.
- G. Pataki. On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues. Mathematics of Operations Research, 23(2):339–358, 1998.
- SE-Sync: A certifiably correct algorithm for synchronization over the special euclidean group. The International Journal of Robotics Research, 38(2-3):95–125, 2019.
- A. Singer. Angular synchronization by eigenvectors and semidefinite programming. Applied and Computational Harmonic Analysis, 30(1):20–36, 2011.
- A. Singer et al. Mathematics for cryo-electron microscopy. Proceedings of the International Congress of Mathematicians (ICM), 3:3981–4000, 2018.
- A. Singer and Y. Shkolnisky. Three-dimensional structure determination from common lines in cryo-em by eigenvectors and semidefinite programming. SIAM Journal on Imaging Sciences, 4(2):543–572, 2011.
- Complete dictionary recovery over the sphere i: Overview and the geometric picture. IEEE Transactions on Information Theory, 63(2):853–884, 2016.
- A geometric analysis of phase retrieval. Foundations of Computational Mathematics, 18(5):1131–1198, 2018.
- Dense networks that do not synchronize and sparse ones that do. Chaos: An Interdisciplinary Journal of Nonlinear Science, 30(8), 2020.
- J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12:389–434, 2012.
- R. Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge University Press, 2018.
- M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint, volume 48. Cambridge University Press, 2019.
- I. Waldspurger and A. Waters. Rank optimality for the Burer–Monteiro factorization. SIAM Journal on Optimization, 30(3):2577–2602, 2020.
- Non-convex exact community recovery in stochastic block model. Mathematical Programming, 195(1-2):1–37, 2022.
- Z. Wen and W. Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142(1-2):397–434, 2013.
- Y. Wu and J. Xu. Statistical inference on graphs: Selected topics, 2023.
- Y. Zhong and N. Boumal. Near-optimal bounds for phase synchronization. SIAM Journal on Optimization, 28(2):989–1016, 2018.