Extension of the surrogate-loss framework to nonconvex surrogates

Determine whether the surrogate-loss framework that models Polyak stepsize updates as gradient descent on the surrogate function ψ(x) = 1/2 h^2(x) with convex h:R^d→R≥0 can be extended to cases where the surrogate h is nonconvex.

Background

The paper presents a unified perspective in which the Polyak stepsize and its variants are interpreted as gradient descent on a surrogate objective ψ(x)=1/2 h^2(x), with h chosen to be convex and nonnegative. This viewpoint explains adaptivity through local curvature properties and supports a range of convergence guarantees.

In discussing limitations, the authors note that their analysis assumes convexity of the surrogate h a priori. Whether similar guarantees and insights extend to nonconvex surrogates remains unresolved, raising the question of applicability of the surrogate-based approach in broader nonconvex optimization settings.