Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Generalized Orthogonal Procrustes Problem under Arbitrary Adversaries (2106.15493v3)

Published 29 Jun 2021 in cs.IT, math.IT, math.OC, and stat.ML

Abstract: The generalized orthogonal Procrustes problem (GOPP) plays a fundamental role in several scientific disciplines including statistics, imaging science and computer vision. Despite its tremendous practical importance, it is generally an NP-hard problem to find the least squares estimator. We study the semidefinite relaxation (SDR) and an iterative method named generalized power method (GPM) to find the least squares estimator, and investigate the performance under a signal-plus-noise model. We show that the SDR recovers the least squares estimator exactly and moreover the generalized power method with a proper initialization converges linearly to the global minimizer to the SDR, provided that the signal-to-noise ratio is large. The main technique follows from showing the nonlinear mapping involved in the GPM is essentially a local contraction mapping and then applying the well-known Banach fixed-point theorem finishes the proof. In addition, we analyze the low-rank factorization algorithm and show the corresponding optimization landscape is free of spurious local minimizers under nearly identical conditions that enables the success of SDR approach. The highlight of our work is that the theoretical guarantees are purely algebraic and do not assume any statistical priors of the additive adversaries, and thus it applies to various interesting settings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Decoding binary node labels from censored edge measurements: Phase transition and efficient recovery. IEEE Transactions on Network Science and Engineering, 1(1):10–22, 2014.
  2. Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2009.
  3. Blind deconvolution using convex programming. IEEE Transactions on Information Theory, 60(3):1711–1732, 2013.
  4. Tightness of the maximum likelihood semidefinite relaxation for angular synchronization. Mathematical Programming, 163(1-2):145–167, 2017.
  5. Multireference alignment using semidefinite programming. In Proceedings of the 5th Conference on Innovations in Theoretical Computer Science, pages 459–470, 2014.
  6. Approximating the little Grothendieck problem over the orthogonal and unitary groups. Mathematical Programming, 160(1):433–475, 2016.
  7. Open problem: Tightness of maximum likelihood semidefinite relaxations. In Conference on Learning Theory, pages 1265–1267. PMLR, 2014.
  8. Method for registration of 3D shapes. In Sensor Fusion IV: Control Paradigms and Data Structures, volume 1611, pages 586–606. International Society for Optics and Photonics, 1992.
  9. N. Boumal. A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints. arXiv preprint arXiv:1506.00575, 2015.
  10. N. Boumal. Nonconvex phase synchronization. SIAM Journal on Optimization, 26(4):2355–2377, 2016.
  11. Deterministic guarantees for burer-monteiro factorizations of smooth semidefinite programs. Communications on Pure and Applied Mathematics, 73(3):581–608, 2020.
  12. S. Burer and R. D. Monteiro. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Mathematical Programming, 95(2):329–357, 2003.
  13. S. Burer and R. D. Monteiro. Local minima and convergence in low-rank semidefinite programming. Mathematical Programming, 103(3):427–444, 2005.
  14. Phase retrieval via matrix completion. SIAM Review, 57(2):225–251, 2015.
  15. Phase retrieval via Wirtinger flow: Theory and algorithms. IEEE Transactions on Information Theory, 61(4):1985–2007, 2015.
  16. E. J. Candès and B. Recht. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717–772, 2009.
  17. Global registration of multiple point clouds using semidefinite programming. SIAM Journal on Optimization, 25(1):468–501, 2015.
  18. Y. Chen and E. J. Candès. The projected power method: An efficient algorithm for joint alignment from pairwise differences. Communications on Pure and Applied Mathematics, 71(8):1648–1714, 2018.
  19. C. Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation III. SIAM Journal on Numerical Analysis, 7(1):1–46, 1970.
  20. N. Dym and Y. Lipman. Exact recovery with symmetries for Procrustes matching. SIAM Journal on Optimization, 27(3):1513–1530, 2017.
  21. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998.
  22. Matrix completion has no spurious local minimum. In Proceedings of the 30th International Conference on Neural Information Processing Systems, pages 2981–2989, 2016.
  23. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6):1115–1145, 1995.
  24. J. C. Gower. Generalized Procrustes analysis. Psychometrika, 40(1):33–51, 1975.
  25. Procrustes Problems, volume 30. Oxford University Press on Demand, 2004.
  26. A. Granas and J. Dugundji. Fixed Point Theory. Springer, 2003.
  27. Matrix completion from a few entries. IEEE Transactions on Information Theory, 56(6):2980–2998, 2010.
  28. Y. Khoo and A. Kapoor. Non-iterative rigid 2D/3D point-set registration using semidefinite programming. IEEE Transactions on Image Processing, 25(7):2956–2970, 2016.
  29. R.-C. Li. New perturbation bounds for the unitary polar factor. SIAM Journal on Matrix Analysis and Applications, 16(1):327–332, 1995.
  30. S. Ling. Improved performance guarantees for orthogonal group synchronization via generalized power method. SIAM Journal on Optimization, 32(2):1018–1048, 2022.
  31. S. Ling. Near-optimal performance bounds for orthogonal and permutation group synchronization via spectral methods. Applied and Computational Harmonic Analysis, 60:20–52, 2022.
  32. S. Ling. Local geometry determines global landscape in low-rank factorization for synchronization. arXiv preprint arXiv:2311.18670, 2023.
  33. S. Ling. Near-optimal bounds for generalized orthogonal procrustes problem via generalized power method. Applied and Computational Harmonic Analysis, 66:62–100, 2023.
  34. S. Ling. Solving orthogonal group synchronization via convex and low-rank optimization: Tightness and landscape analysis. Mathematical Programming, 200(1):589–628, 2023.
  35. On the estimation performance and convergence rate of the generalized power method for phase synchronization. SIAM Journal on Optimization, 27(4):2426–2446, 2017.
  36. A unified approach to synchronization problems over subgroups of the orthogonal group. Applied and Computational Harmonic Analysis, 2023.
  37. Implicit regularization in nonconvex statistical estimation: Gradient descent converges linearly for phase retrieval, matrix completion, and blind deconvolution. Foundations of Computational Mathematics, 20:451–632, 2020.
  38. Point registration via efficient convex relaxation. ACM Transactions on Graphics (TOG), 35(4):1–12, 2016.
  39. Registration of point cloud data from a geometric optimization perspective. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing, pages 22–31, 2004.
  40. Efficient rounding for the noncommutative Grothendieck inequality. In Proceedings of the Forty-fifth Annual ACM Symposium on Theory of Computing, pages 71–80, 2013.
  41. A. Nemirovski. Sums of random symmetric matrices and quadratic optimization under orthogonality constraints. Mathematical Programming, 109(2):283–317, 2007.
  42. A survey of structure from motion. Acta Numerica, pages 305–364, 2017.
  43. The generalized orthogonal Procrustes problem in the high noise regime. Information and Inference: A Journal of the IMA, 2021.
  44. SE-Sync: A certifiably correct algorithm for synchronization over the special Euclidean group. The International Journal of Robotics Research, 38(2-3):95–125, 2019.
  45. P. H. Schönemann. A generalized solution of the orthogonal Procrustes problem. Psychometrika, 31(1):1–10, 1966.
  46. A. Singer et al. Mathematics for cryo-electron microscopy. Proceedings of the International Congress of Mathematicians (ICM), 3:3981–4000, 2018.
  47. A. Singer and Y. Shkolnisky. Three-dimensional structure determination from common lines in cryo-em by eigenvectors and semidefinite programming. SIAM Journal on Imaging Sciences, 4(2):543–572, 2011.
  48. A. M.-C. So. Moment inequalities for sums of random matrices and their applications in optimization. Mathematical Programming, 130(1):125–151, 2011.
  49. A brief introduction to statistical shape analysis. Informatics and Mathematical Modelling, Technical University of Denmark, DTU, 15(11), 2002.
  50. Complete dictionary recovery over the sphere i: Overview and the geometric picture. IEEE Transactions on Information Theory, 63(2):853–884, 2016.
  51. A geometric analysis of phase retrieval. Foundations of Computational Mathematics, 18(5):1131–1198, 2018.
  52. J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12:389–434, 2012.
  53. R. Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge University Press, 2018.
  54. M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint, volume 48. Cambridge University Press, 2019.
  55. I. Waldspurger and A. Waters. Rank optimality for the Burer–Monteiro factorization. SIAM Journal on Optimization, 30(3):2577–2602, 2020.
  56. L. Wang and A. Singer. Exact and stable recovery of rotations for robust synchronization. Information and Inference: A Journal of the IMA, 2(2):145–193, 2013.
  57. Optimal non-convex exact recovery in stochastic block model via projected power method. In International Conference on Machine Learning, pages 10828–10838. PMLR, 2021.
  58. P.-Å. Wedin. Perturbation bounds in connection with singular value decomposition. BIT Numerical Mathematics, 12(1):99–111, 1972.
  59. Z. Wen and W. Yin. A feasible method for optimization with orthogonality constraints. Mathematical Programming, 142(1):397–434, 2013.
  60. Orthogonal trace-sum maximization: Tightness of the semidefinite relaxation and guarantee of locally optimal solutions. SIAM Journal on Optimization, 32(3):2180–2207, 2022.
  61. Y. Zhong and N. Boumal. Near-optimal bounds for phase synchronization. SIAM Journal on Optimization, 28(2):989–1016, 2018.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now