First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems (1706.06461v1)

Abstract: We focus on nonconvex and nonsmooth minimization problems with a composite objective, where the differentiable part of the objective is freed from the usual and restrictive global Lipschitz gradient continuity assumption. This longstanding smoothness restriction is pervasive in first order methods (FOM), and was recently circumvent for convex composite optimization by Bauschke, Bolte and Teboulle, through a simple and elegant framework which captures, all at once, the geometry of the function and of the feasible set. Building on this work, we tackle genuine nonconvex problems. We first complement and extend their approach to derive a full extended descent lemma by introducing the notion of smooth adaptable functions. We then consider a Bregman-based proximal gradient methods for the nonconvex composite model with smooth adaptable functions, which is proven to globally converge to a critical point under natural assumptions on the problem's data. To illustrate the power and potential of our general framework and results, we consider a broad class of quadratic inverse problems with sparsity constraints which arises in many fundamental applications, and we apply our approach to derive new globally convergent schemes for this class.

Analysis of First Order Methods beyond Convexity and Lipschitz Gradient Continuity

The paper "First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems" presents significant advancements in optimization through the development of novel first-order methods that operate beyond the typical convexity and Lipschitz gradient continuity assumptions. The research explores nonconvex and nonsmooth minimization challenges, particularly those involving a composite objective. Importantly, the paper circumvents the traditional requirement for global Lipschitz gradient continuity in first-order methods by expanding previous frameworks developed for convex composite optimization to encompass genuinely nonconvex problems.

Fundamental Contributions

1. Extended Framework for Nonconvex Problems: Building on the work of Bauschke, Bolte, and Teboulle, the authors have extended the framework of smooth adaptable functions to nonconvex settings. This allows for the derivation of a full extended Descent Lemma, substituting conventional quadratic approximation with more general proximity measures that better capture the function's and feasible set's geometry.
2. Novel Bregman-Based Proximal Gradient Methods: Introducing Bregman proximal gradient methods for nonconvex composite models with smooth adaptable functions, the paper proves global convergence to a critical point under certain assumptions. The methods presented are notable for their departure from conventional assumptions, deploying non-Euclidean distances of the Bregman type.
3. Application to Quadratic Inverse Problems: The research effectively demonstrates the utility of its framework by addressing a broad class of quadratic inverse problems that include sparsity constraints, an important concern in numerous practical applications like phase retrieval and signal processing.

Numerical Results and Claims

The paper provides a robust theoretical grounding, showcasing global convergence without necessitating a globally Lipschitz continuous gradient—a notable departure from existing methodologies. Furthermore, the researchers illustrate this with quadratic inverse problems where the algorithms derived via their approach are provably the first to guarantee global convergence, marking significant progress in optimization theory beyond traditional confines.

Implications and Future Prospects

The implications of these findings extend both practically and theoretically. Practically, these methods contribute to fields where signal recovery and phase retrieval are paramount, suggesting potential applications in medical imaging, telecommunications, and more. Theoretically, by overcoming longstanding assumptions, the work encourages deeper exploration into nonconvex optimization landscapes which may lead to breakthrough capabilities in diverse computational settings.

The prospect of extending these methodologies further suggests potential advancements in areas such as machine learning, where nonconvex optimization challenges are prevalent, and improved algorithmic efficiency and outcomes can significantly affect model training and deployment.

Conclusion

The paper presents an important step forward in optimization theory, expanding the boundaries of first-order methods beyond traditional assumptions. Its combination of theoretical rigor, practical applicability, and potential for future research contributions positions it as a valuable resource for researchers engaged in optimization and applied mathematics.