Ridge Kernel Averaging and Uniform Approximation

Abstract: We develop a framework for function classes generated by parametric ridge kernels: one-dimensional kernels composed with affine projections and averaged over a parameter measure. The induced kernels are positive definite, and the resulting integral class coincides isometrically with its reproducing kernel Hilbert space. We characterize all kernels obtainable by varying the measure as the uniform closure of the conic hull of ridge atoms, giving a sharp universality criterion; a slice-wise polynomial versus non-polynomial dichotomy governs expressivity. We then analyze random-kernel networks whose activations may be indefinite but have a positive-definite mean. For these networks we prove a Monte Carlo rate with mean-squared error of order one over $N$ and a high-probability uniform bound on compact sets, without requiring pathwise positive definiteness.