A heavy-ball type curve search method for smooth convexly constrained optimization

Abstract: This paper addresses smooth convexly constrained optimization problems where the Euclidean projection onto the feasible set is computationally tractable. Although momentum techniques like Polyak's heavy-ball are known for accelerating optimization algorithms, their use in constrained settings remains limited due to challenges in preserving feasibility and ensuring convergence. We thus propose a heavy-ball-type method that extends to the constrained case a recently introduced curve-search globalization framework. The method attempts a momentum update and performs a curvilinear search to enforce an Armijo-type descent condition: when the momentum step is infeasible or unacceptable, the algorithm smoothly reverts to a feasible descent direction. We prove that the algorithm is well-defined and globally convergent to stationary points; the derivation of these results is nontrivial due to the use of a heavy-ball type direction in a constrained setting, where it may generate infeasible iterates. We discuss the incorporation of further mechanisms into the algorithm, including non-monotone curve search, spectral steplength selection and an adaptive momentum strategy. Numerical experiments on benchmark problems show the method is robust and competitive with the state-of-the-art.