---
title: Non-Stationary Isotropic Kernels
url: https://www.emergentmind.com/topics/non-stationary-isotropic-kernels
type: topic
---

# Non-Stationary Isotropic Kernels

Non-stationary isotropic kernels are covariance functions defined on Euclidean space $\mathbb{R}^d$ that are invariant under the action of the orthogonal group $O(d)$ but whose dependence on location is not restricted to stationary differences (i.e., they are not translation invariant). Such kernels combine the geometric invariance of isotropy—covariance functions that depend only on the “shape” of pairs $(x, y)$ modulo rotations—with the modeling flexibility of non-stationarity, permitting spatially heterogeneous scaling, amplitude, and regularity. This framework encompasses and unifies classical stationary isotropic kernels, dot product kernels, and their non-stationary generalizations, and includes prominent machine learning objects such as infinite-width neural network kernels.

## 1. Mathematical Characterization of Non-Stationary Isotropic Kernels

Let $K: \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}$ be a continuous kernel satisfying $K(x, y) = K(Ux, Uy)$ for all $U \in O(d)$. The principal result is that any such kernel admits an expansion of the form
\[
K(x, y) = \sum_{n=0}^{\infty} \alpha_n^{(d)}(\|x\|, \|y\|) \, \widetilde{P}_n(\langle x/\|x\|,\, y/\|y\|\rangle),
\]
where
- $\alpha_n^{(d)}: [0, \infty) \times [0, \infty) \to \mathbb{R}$ are continuous “radial” kernels, uniquely determined by $K$, and must be such that $\sum_n \alpha_n^{(d)}(r, r) < \infty$ for all $r$,
- $\widetilde{P}_n$ is the degree-$n$ normalized Gegenbauer polynomial with index $\lambda = (d-2)/2$.

This structure generalizes Schoenberg’s classical result for inner-product–dependent kernels on the sphere (where the $\alpha_n^{(d)}$ reduce to constants). When $K$ is stationary isotropic, the dependence on $\|x\|,\|y\|$ becomes a function of $r = \|x-y\|$, while restriction to the unit sphere ($\|x\| = \|y\|=1$) recovers dot product kernels [2506.22048].

## 2. Strict Positive Definiteness

For $K$ as above to be strictly positive definite, two requirements must be met:
- The “base” kernel must be positive at the origin: $\alpha_0^{(d)}(0,0) > 0$.
- For any finite set $\{r_1, ..., r_m\}$ of distinct norm values, and any nonzero $c \in \mathbb{R}^m$, the quadratic forms $c^\top [\alpha_n^{(d)}(r_i, r_j)]_{i,j=1}^m c$ are nonzero for infinitely many even and infinitely many odd $n$.

This ensures that the kernel captures a sufficiently rich set of geometric “modes” (even and odd degree components) on any set of distinct radial shells. Specializing to dot product kernels (with $\alpha_n^{(d)}(r, s) = a_n (rs)^n$), one recovers the classical requirement that infinitely many even and infinitely many odd $a_n$ coefficients must be positive for strict positive definiteness [2506.22048].

## 3. Unification of Stationary, Dot Product, and Neural Network Kernels

The general expansion simultaneously captures:
- **Stationary Isotropic Kernels**: If $K(x, y) = h(\|x-y\|)$, then $\alpha_n^{(d)}(\|x\|, \|y\|)$ simplifies to a function of $\|x-y\|$, and the kernel depends on $x$ and $y$ only via their separation.
- **Dot Product Kernels**: When restricted to $\|x\| = \|y\| = 1$, the expansion becomes $K(x, y) = \sum_n a_n\, \langle x, y\rangle^n$ (Schoenberg’s sphere theorem).
- **Infinite-Width Neural Network Kernels**: For many architectures (e.g., NNGP, NTK), the covariance kernel between network outputs is of the form
  \[
  K(x, y) = \sum_{m=0}^{\infty} \alpha_m(\|x\|, \|y\|)\langle x/\|x\|,\, y/\|y\|\rangle^m,
  \]
  where $\alpha_m$ are recursively computable functions derived from the activation and initialization in the neural network. These kernels are non-stationary because they depend separately on $\|x\|$ and $\|y\|$, but isotropic because the only geometric dependence is via the normalized inner product [2506.22048].

## 4. Connection to Spatial Statistics and Machine Learning

This unification brings fundamental insight into the design and analysis of kernels in several domains:
- **Gaussian Process Regression and Kernel Methods**: The expansion gives a systematic template for constructing kernels able to express locally adaptive variance and geometry, beyond the restrictions of translation invariance.
- **Spatial Statistics**: Non-stationary isotropic kernels are essential for modeling spatially heterogeneous media, for example where correlation length or amplitude varies with location, but where rotational symmetry is natural.
- **Neural Architecture Analysis**: The realization that wide neural networks induce non-stationary isotropic kernels links deep learning theory with kernel methods, enabling cross-fertilization via harmonic analysis and classical positive-definiteness criteria.

## 5. High-Dimensional and Infinite-Dimensional Regime

The characterization is stable as $d \to \infty$, since Gegenbauer polynomials asymptotically become monomials and the series expansion remains well-defined. This property is critical for high-dimensional function analysis and for understanding the geometry of function spaces associated with neural tangent kernels (NTK) and related GP priors [2506.22048].

## 6. Practical Guidance for Construction and Use

The expansion $K(x, y) = \sum_n \alpha_n^{(d)}(\|x\|, \|y\|)\, \widetilde{P}_n(\langle x/\|x\|, y/\|y\|\rangle)$ provides a concrete recipe:
- One designs or learns the functions $\alpha_n^{(d)}(\cdot,\cdot)$ to encode domain knowledge or fit data, for example adapting to spatial inhomogeneity, input-dependent scaling, or to capture effects of deep learning feature spaces.
- For strict positive definiteness, care must be taken to ensure that both even and odd degree sectors are nontrivial across all radial slices under consideration.

In summary, the Schoenberg-type characterization synthesizes stationary, dot product, and non-stationary isotropic kernel classes into a single positive-definite framework. It enables principled generalizations for spatial statistics, kernel learning, and high-dimensional inference, and precisely delineates the interplay between rotational invariance and local adaptability in non-stationary covariance modeling [2506.22048].

Source: https://www.emergentmind.com/topics/non-stationary-isotropic-kernels