Lazifying point insertion algorithms in spaces of measures

Abstract: Greedy point insertion algorithms have emerged as an attractive tool for the solution of minimization problems over the space of Radon measures. Conceptually, these methods can be split into two phases: first, the computation of a new candidate point via maximizing a continuous function over the spatial domain, and second, updating the weights and/or support points of all Dirac-Deltas forming the iterate. Under additional structural assumptions on the problem, full resolution of the subproblems in both steps guarantees an asymptotic linear rate of convergence for pure coefficient updates, or finite step convergence, if, in addition, the position of all Dirac-Deltas is optimized. In the present paper, we lazify point insertion algorithms and allow for the inexact solution of both subproblems based on computable error measures, while provably retaining improved theoretical convergence guarantees. As a specific example, we present a new method with a quadratic rate of convergence based on Newton steps for the weight-position pairs, which we globalize by point-insertion as well as clustering steps.