Greedy multiscale kernel algorithms

Abstract: Multiscale kernel approximation and greedy kernel algorithms are two established tools for scattered data approximation. We combine them into greedy multiscale kernel algorithms, in which every level of a multiscale residual correction is constructed by a greedy kernel algorithm. For the multiscale $f$-greedy algorithm we prove convergence rates in the number of selected centres which coincide (up to an arbitrarily small loss in the exponent) with the rates of the single-scale $f$-greedy algorithm. The gain of the multiscale construction is in the stability, and we make this precise by a two-sided estimate for the condition number of kernel matrices with rescaled kernels. The lower bound shows that any single-scale method, and in particular the $f$-greedy algorithm, pays for its convergence rate with condition numbers growing at least like a power of the number of centres. In contrast, the multiscale scaling of the kernel keeps this lower bound fixed on every level without changing the rate. The upper bound identifies the separation of the centres at the scale of the kernel as the quantity to control. We give the operation counts of all the methods considered and compare the greedy multiscale algorithms numerically with single-scale greedy and classical multiscale methods, in terms of error, time and conditioning.