A Novel First-order Method with Event-driven Objective Evaluations

Abstract: Arising in semi-parametric statistics, control applications, and as sub-problems in global optimization methods, certain optimization problems can have objective functions requiring numerical integration to evaluate, yet gradient function evaluations that are relatively cheap. For such problems, typical optimization methods that require multiple evaluations of the objective for each new iterate become computationally expensive. In light of this, optimization methods that avoid objective function evaluations are attractive, yet we show anti-convergence behavior for these methods on the problem class of interest. To address this gap, we develop a novel gradient algorithm that only evaluates the objective function when specific events are triggered and propose a step size scheme that greedily takes advantage of the properties induced by these triggering events. We prove that our methodology has global convergence guarantees under the most general smoothness conditions, and show through extensive numerical results that our method performs favorably on optimization problems arising in semi-parametric statistics.