The non-convex Burer-Monteiro approach works on smooth semidefinite programs (1606.04970v3)

Abstract: Semidefinite programs (SDPs) can be solved in polynomial time by interior point methods, but scalability can be an issue. To address this shortcoming, over a decade ago, Burer and Monteiro proposed to solve SDPs with few equality constraints via rank-restricted, non-convex surrogates. Remarkably, for some applications, local optimization methods seem to converge to global optima of these non-convex surrogates reliably. Although some theory supports this empirical success, a complete explanation of it remains an open question. In this paper, we consider a class of SDPs which includes applications such as max-cut, community detection in the stochastic block model, robust PCA, phase retrieval and synchronization of rotations. We show that the low-rank Burer--Monteiro formulation of SDPs in that class almost never has any spurious local optima.

• The paper confirms that non-convex Burer-Monteiro reformulations can achieve global optimality for key classes of smooth semidefinite programs.
• Methodological analysis reveals that under rank constraints, local search methods converge to global solutions, validated by rigorous numerical simulations.
• The study introduces the Riemannian trust-region method, demonstrating practical efficiency and scalability for large SDP instances across diverse applications.

The Non-Convex Burer-Monteiro Approach for Smooth Semidefinite Programs

The paper addresses the challenge of solving semidefinite programs (SDPs) that are often computationally intensive when approached with traditional interior point methods, especially for larger problem sizes. It revives the approach proposed by Burer and Monteiro, which involves rank-restricted, non-convex reformulations of SDPs with few equality constraints. This approach intriguingly suggests that local optimization methods can converge reliably to global optima, despite the non-convex nature of the problem.

Key Contributions

1. Theoretical Insights on Non-Convex Reformulations: The paper affirms that for a significant class of SDPs—including problems like max-cut and various synchronization and community detection problems—these non-convex surrogates nearly always lack spurious local optima. This is significant as it provides an unusual theoretical guarantee in the field of non-convex optimization, indicating that local optimality conditions may suffice for global optimality for these problems.
2. Numerical Validation: The authors underscore their theoretical claims with numerical simulations, showcasing that the Burer--Monteiro formulation has practical implementations that not only preserve the global solution properties of the original SDP but also scale more efficiently.
3. Rank Sufficiency: The paper delineates conditions under which the reformulation works effectively, specifically highlighting the role of rank restriction. They prove that when the rank parameter $p$ satisfies the inequality $\frac{p(p+1)}{2} > m$, the reformulated non-convex problem possesses robust global optimality guarantees for almost all cost matrices $C$.
4. Importance of the Manifold Structure: The paper also emphasizes the manifold structure of the rank-restricted space and the importance of regularity conditions or constraint qualifications to assure that the manifold assumptions hold across the feasible set consistently.
5. Practical Algorithms: Furthermore, the authors introduce the Riemannian trust-region method (RTR) as a viable computational tool to solve these reformulated SDPs, with provable convergence to second-order critical points, regardless of initialization.

Implications and Future Directions

• Practical Performance: The paper demonstrates that using the Burer--Monteiro approach allows the handling of larger SDP instances more efficiently, which will be crucial in scientific and engineering contexts characterized by extensive data.
• Convergence in Non-Convex Settings: The results break new ground in understanding how certain non-convex problems can be approached with confidence that global solutions are attainable. This insight could stimulate further exploration in other areas of optimization where non-convexity is traditionally seen as a daunting challenge.
• Applications Beyond Max-Cut and Synchronization: While the paper focuses on several specific problem types, the framework they have built has potential applicability to a broader class of problems, particularly where SDPs manifest in control, signal processing, and machine learning.

Future research could expand on this approach, investigating its applicability to broader classes of SDPs, including those with inequalities and exploring robust numerical methods that could take full advantage of this reformulation's theoretical strengths. Additionally, there could be further work to simplify or relax the regularity assumptions required for the results to hold, thus broadening the applicability of the theoretical breakthroughs made in this paper.

In conclusion, this investigation into the Burer--Monteiro approach sheds light on the latent global nature of low-rank factorized SDP solutions, offering both theoretical and empirical assurances that bolster its utility in varied contexts where traditional SDP solvers falter. As real-world applications continue to scale, such results are paramount for continuous advancements in optimization methodologies that underpin modern computational paradigms.