Convergence of Pivoted Cholesky Algorithm for Lipschitz Kernels

Abstract: We investigate the continuous analogue of the Cholesky factorization, namely the pivoted Cholesky algorithm. Our analysis establishes quantitative convergence guarantees for kernels of minimal smoothness. We prove that for a symmetric positive definite Lipschitz continuous kernel $K:\Omega\times \Omega \rightarrow \mathbb{R}$ on a compact domain $\Omega\subset\mathbb{R}^d$, the residual of the Cholesky algorithm with any pivoting strategy is uniformly bounded above by a constant multiple of the fill distance of pivots. In particular, our result implies that under complete pivoting (where the maximum value of the diagonal of the residual is selected as the next pivot): \begin{equation*} |R_n|{\infty} = \mathcal{O}(n^{-1/d}), \end{equation*} where $R_n$ is the residual after $n$ Cholesky steps and $|\cdot|\infty$ is the absolute maximum value of $R_n$. Moreover, if $K$ is differentiable in both variables with a Lipschitz derivative, our convergence rate improves to $\mathcal{O}(n^{-2/d})$. Our result closes a gap between theory and practice as previous analyses required $C^2$-regularity of $K$ to establish convergence, whereas empirical evidence indicated robust performance even for non-differentiable kernels. We further detail how our convergence results propagate to downstream applications, including discrete analogues, Gaussian process regression, and the P-greedy interpolation method.