A Modified Proximal Bundle Method Under A Frank-Wolfe Perspective (2411.15926v2)

Abstract: The proximal bundle method (PBM) is a fundamental and computationally effective algorithm for solving optimization problems with nonsmooth components. In this paper, we conduct a theoretical investigation of a modified proximal bundle method, which we call the Modified Proximal Bundle with Fixed Absolute Accuracy (MPB-FA). MPB-FA modifies PBM in two key aspects. Firstly, the null-step test of MPB-FA is based on an absolute accuracy criterion, and the accuracy is fixed over iterations, while the standard PBM uses a relative accuracy in the null-step test, which changes with iterations. Secondly, the proximal parameter in MPB-FA is also fixed over iterations, while it is permitted to change in the standard PBM. These modifications allow us to interpret a sequence of null steps of MPB-FA as a Frank-Wolfe algorithm on the Moreau envelope of the dual problem. In light of this correspondence, we first extend the linear convergence of Kelley's method on convex piecewise linear functions from the positive homogeneous to the general case. Building on this result, we propose a novel complexity analysis of MPB-FA and derive an $\mathcal{O}(\epsilon^{-4/5})$ iteration complexity, improving upon the best known $\mathcal{O}(\epsilon^{-2})$ guarantee on a related variant of PBM. It is worth-noting that the best known complexity bound for the classic PBM is $\mathcal{O}(\epsilon^{-3})$. Our approach also reveals new insights on bundle management.