Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey (1507.01030v2)

Abstract: We survey incremental methods for minimizing a sum $\sum_{i=1}^mf_i(x)$ consisting of a large number of convex component functions $f_i$. Our methods consist of iterations applied to single components, and have proved very effective in practice. We introduce a unified algorithmic framework for a variety of such methods, some involving gradient and subgradient iterations, which are known, and some involving combinations of subgradient and proximal methods, which are new and offer greater flexibility in exploiting the special structure of $f_i$. We provide an analysis of the convergence and rate of convergence properties of these methods, including the advantages offered by randomization in the selection of components. We also survey applications in inference/machine learning, signal processing, and large-scale and distributed optimization.

• The paper introduces a unified framework for incremental methods that efficiently minimizes convex functions by processing one component at a time.
• It presents rigorous convergence analyses for both cyclic and randomized component selections, emphasizing improved error bounds and rates.
• The survey highlights practical applications in machine learning and signal processing while paving the way for adaptive stepsize and asynchronous strategies.

Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: An Expert Overview

This paper presents a comprehensive survey of incremental optimization methods for minimizing a sum of convex component functions, that have emerged as indispensable tools in the field of large-scale optimization. The author, Dimitri P. Bertsekas, provides a structured analysis focusing on scenarios where the optimization objective is decomposed into a significant number of component functions $(f_i(x))$. The methods discussed, which include incremental gradient, subgradient, and proximal methods, can effectively exploit the structure of composite objectives, offering practical advantages in both convergence rate and computational efficiency.

Algorithmic Framework

The paper introduces a unified framework for a range of incremental optimization techniques. The goal is to minimize the sum of component functions by updating the solution incrementally, processing one component function at a time rather than the entire objective. Such an approach is not only computationally advantageous when the number of component functions, $m$, is large, but also allows for flexibility through the use of combinations between gradient/subgradient and proximal methods.

Incremental methods in this survey include:

• Incremental Gradient Methods: Addressing differentiable problems, these methods update iterates based on the gradient of a single component at a time.
• Incremental Subgradient Methods: Extending applicability to nondifferentiable convex functions, this variant uses subgradients.
• Incremental Proximal Methods: Leveraging proximal iterations which are preferable when a certain structure can be exploited in the component functions.

Further, the paper proposes hybrid algorithms combining subgradient and proximal updates, which are shown to provide additional flexibility and utility in solving complex structured convex problems.

Convergence Analysis

Significant effort is dedicated to the convergence properties of these incremental methods. The analysis is bifurcated into two main scenarios:

• Cyclic Order of Component Selection: Here, the sequence of components is iterated in a fixed order cycle, and the paper provides conditions under which convergence to an optimal solution is guaranteed, even if the stepsize is kept constant.
• Randomized Component Selection: This approach involves selecting the component functions at random, with analysis showing that randomization can lead to better convergence properties compared to deterministic cyclical order, particularly by reducing the risk of unfavorable component selection orders.

Key results include proofs of convergence to a neighborhood of the optimal solution for constant stepsizes, with the error bound diminishing as the stepsize is reduced. More intriguingly, when utilizing randomization in selection, the incremental methods achieve convergence behavior with improved rates over deterministic counterparts.

Application Contexts and Implications

Incremental methods shine in various applications ranging from machine learning and signal processing to large-scale distributed optimization problems. The survey elaborates on their use in practical contexts, such as:

• Regularized Least Squares: Incremental algorithms prove particularly efficacious for problems featuring $\ell_1$ regularization, a popular method in compressed sensing and sparse recovery.
• Iterated Projection Algorithms: For problems involving feasibility by iterating projections onto convex sets, incremental methods provide effective solutions when faced with many constraints.

The survey also discusses extensions and modifications, such as nonquadratic proximal terms and distributed/asynchronous implementations, which promise to further expand the scope of potential applications.

Conclusion

The paper underlines the significance of incremental methods in addressing the computational demands of modern optimization problems characterized by large-scale and distributed data sets. By presenting a unified theory and comparative insights, Bertsekas not only bridges various established methods but also sets a foundational platform for future research and development in improving convergence rates and addressing new problem classes. Subsequent research can fruitfully explore adaptive stepsize strategies and explore novel applications in more complex environments, reinforcing the practical utility and theoretical richness of incremental optimization methods.