---
title: Gaussian and Polynomial Kernels
url: https://www.emergentmind.com/topics/gaussian-and-polynomial-kernels
type: topic
---

# Gaussian and Polynomial Kernels

Gaussian and polynomial kernels are foundational classes of positive definite kernels central to machine learning, approximation theory, and computational physics. These kernels induce concrete feature maps into (potentially infinite-dimensional) Hilbert spaces, yielding flexible hypothesis classes for regression, classification, and surrogate modeling. Their algebraic structure, universal approximation properties, connections via polynomial expansions, asymptotic regimes, and efficient computation underlie much of the recent progress in scalable kernel methods, functional data analysis, and surrogate modeling for complex scientific problems.

## 1. Definitions, Feature Maps, and Algebraic Structure

The most widely used kernels on $\mathbb{R}^D$ are:

- **Gaussian (RBF) kernel** (universal):
  $$
  k_{\rm G}(x, x') = \exp\left(-\frac{\|x - x'\|^2}{2\sigma^2}\right)
  $$
  where $\|\cdot\|$ is Euclidean norm, and $\sigma > 0$ is the length-scale (bandwidth).

- **Polynomial kernel** (order $p$; not universal):
  $$
  k_{\rm poly}(x, x') = (c + x^\top x')^p
  $$
  with $c \ge 0$ (often $c=1$) controlling the offset, and $p \in \mathbb{N}$ the degree.

Both are positive-definite; the induced feature map $\phi(\cdot)$ satisfies $k(x, x') = \langle \phi(x), \phi(x') \rangle_\mathcal{H}$. For the polynomial kernel, the feature map enumerates all monomials of total degree $\le p$, and the feature-space dimension is $\binom{p+D}{D}$. The Gaussian kernel, by contrast, admits an infinite-dimensional feature space, often represented via the Hermite expansion or random Fourier features [2103.17233][1109.4603][1503.00021].

## 2. Taylor Expansions and the Gaussian-Polynomial Connection

The Gaussian kernel’s analytic structure allows for polynomial approximations via its Taylor (Mercer) expansion in the inner product:

$$
k_{\rm G}(x, x') = \exp\left(-\frac{\|x\|^2 + \|x'\|^2}{2\sigma^2}\right) \cdot \exp\left(\frac{x^\top x'}{\sigma^2}\right) = e^{-\frac{\|x\|^2+\|x'\|^2}{2\sigma^2}} \sum_{m=0}^\infty \frac{1}{m!}\left(\frac{x^\top x'}{\sigma^2}\right)^m
$$

Truncating at $Q$ yields a finite-dimensional polynomial kernel approximation. Each term is an inner product of degree-$m$ monomials, giving a hierarchy of polynomial spaces, and the truncation error for $\|x\|, \|x'\| \le R$ is bounded by $(R^2/\sigma^2)^{Q+1}/(Q+1)!$ [1109.4603]. For $R^2/\sigma^2 = O(1)$, only $O(\log n)$ terms are required to achieve error $\epsilon$ on datasets of size $n$, yielding efficient polynomial approximations to the Gaussian kernel [2108.09420][1909.01410].

## 3. Approximation Power and Limiting Regimes

Polynomial kernels are non-universal: the associated RKHS comprises polynomials up to degree $p$, with rapid convergence to zero error after $p+1$ samples in general position. In contrast, the Gaussian kernel’s RKHS is dense in $C(\mathcal{X})$ (for compact $\mathcal{X}$), achieving super-exponential approximation rates:

$$
e_n^{\min} \sim \left(\frac{\varepsilon}{2}\right)^n (n!)^{-1/2}
$$

for $n$ samples and width parameter $\varepsilon$ [2209.12473]. However, in high-dimensional, sparse-data regimes, the optimal Gaussian bandwidth diverges, and the kernel regresses to low-degree polynomials—formally, for $\ell \gg$ data diameter, the Gaussian kernel’s expansion truncates after finitely many terms, and kernel regression reduces to polynomial regression [2311.10790]. This regime transition is rigorously characterized: if optimal $\ell^* \gtrsim D N^{1/d}$ for $d$-dimensional data, only $O(1)$ degrees matter regardless of $N$ [2311.10790].

The minimal polynomial degree required for uniform approximation of $e^{-x}$ to error $\delta$ on $[0,B]$ is $d_{B;\delta}(e^{-x}) = \Theta(\max\{\sqrt{B\log(1/\delta)},\ \frac{\log(1/\delta)}{\log(B^{-1}\log(1/\delta))}\})$ [2205.06249].

## 4. Symmetric and Antisymmetric Kernel Lifting

For tasks requiring invariance or equivariance to permutations (e.g., indistinguishable particles, point-cloud data in quantum chemistry), standard kernels can be lifted to symmetric (invariant) or antisymmetric (sign-changing) forms via averaging over the symmetric group $S_D$:

- **Symmetrized kernel:**
  $$
  k_s(x, x') = \frac{1}{D!} \sum_{\pi \in S_D} k\!\bigl(\pi(x), x'\bigr)
  $$
- **Antisymmetrized kernel:**
  $$
  k_a(x, x') = \frac{1}{D!} \sum_{\pi \in S_D} \sgn(\pi)\,k\!\bigl(\pi(x), x'\bigr)
  $$

These remain positive-definite. Symmetrized polynomial kernels have feature spaces indexed by integer partitions, reducing dimension and thus sample complexity compared to the conventional tensor basis. Antisymmetric polynomial kernels require all monomial exponents to be distinct; for the Gaussian kernel, the antisymmetrization yields the Slater-determinant representation:

$$
k_a(x, x') = \frac{1}{D!}\det \left( \exp\left(-\frac{(x_i - x'_j)^2}{2\sigma^2}\right) \right)_{i,j=1}^D
$$

which is standard in many-fermion wavefunction modeling and can be evaluated in $O(D^3)$ time [2103.17233].

Imposing (anti)symmetry reduces the hypothesis space dimension and can halve or further reduce the necessary training data, improving data efficiency and conditioning in physics and chemistry applications.

## 5. Efficient Computation: Oblivious Sketching and Feature Approximations

Naïve kernel methods scale poorly with sample size $n$ and feature dimension, especially for high-degree polynomial or infinite-dimensional Gaussian kernels. Recent advances use oblivious sketching—combining fast Johnson–Lindenstrauss (SRHT) and TensorSketch methods—to embed the lifted polynomial feature space into $\mathbb{R}^m$, with $m$ only polynomial in $q$ (degree) and dataset size, independently of $d^q$ [2108.09420][1909.01410].

For the Gaussian kernel, Taylor truncation plus polynomial sketch achieves kernel inner-product approximation in $O(nd + \epsilon^{-2}n^2 \operatorname{polylog} n)$ time, eliminating the linear-in-degree slowdown typical of prior approaches. Similarly, random Fourier features and explicit monomial “Taylor features” provide alternative approximations: for $s$-sparse data and moderate $M$, Taylor features often outperform random Fourier features per FLOP for bandwidths not much smaller than the data radius [1109.4603]. Sketching enables kernel ridge regression, SVMs, and kernel PCA to operate efficiently at large scale.

## 6. RKHS, Mercer Expansions, and Operator-Theoretic Perspective

Both Gaussian and polynomial kernels admit Mercer (eigenfunction) expansions:

- **Gaussian kernel on compact $\mathcal{X}$:**
  $$
  k_G(x, x') = \sum_{i=1}^\infty \lambda_i^{(G)} \varphi_i(x) \varphi_i(x')
  $$
  with super-exponential eigenvalue decay.

- **Polynomial kernel (degree $p$):**
  $$
  k_P(x, x') = \sum_{|\alpha| = p} \lambda_\alpha \psi_\alpha(x) \psi_\alpha(x')
  $$
  with finitely many nonzero eigenvalues, corresponding to the monomial basis of order $p$.

Truncating the Gaussian Mercer series reduces it to polynomial kernels, and in the “flat-kernel” ($\sigma \to \infty$) regime, the induced RKHS converges to the polynomial subspace of degree determined by the sample configuration [1906.02096][2209.12473]. The Gaussian kernel’s infinite smoothness yields super-exponential convergence for function approximation, while finite-rank polynomial kernels offer algebraic rates and exact interpolation upon sufficient sampling.

Kernel ridge regression with a truncated Gaussian Mercer expansion is equivalent to pseudospectral polynomial projection, provided the data locations form an exact quadrature rule for the polynomial space—establishing direct equivalence between Gaussian-process regression and polynomial least-squares in this regime [1503.00021][1906.02096].

## 7. Summary Table: Key Properties

| Kernel Type           | RKHS Dimension          | Universality         | Approximation Rate         | Efficient Sketch/Ftrs                  |
|-----------------------|------------------------|----------------------|----------------------------|----------------------------------------|
| Polynomial ($p$)      | $\binom{p+D}{D}$       | Non-universal        | Exact after $p+1$ samples  | Oblivious sketching, explicit monomials|
| Gaussian (RBF)        | Infinite               | Universal            | Super-exponential (factorial in $n$) | Taylor features, random Fourier, sketching |
| Sym. Poly.            | integer partitions     | Non-universal        | Reduced monomial count     | Permutationally invariant polynomial bases |
| Antisym. Poly./Gauss. | partition constraints  | Universal on antisymmetric subspaces | Reduced monomials/Slater det. | Slater determinant, sketching           |

The interplay of analytic structure, spectral decay, permutation invariance, and computational approximation governs the selection and deployment of Gaussian and polynomial kernels across scientific and data-driven domains. Key algorithmic advances have made scalable kernel methods feasible for a range of large-scale and high-dimensional problems, while deep operator-theoretic and algebraic connections clarify regimes in which each kernel class is most effective [2103.17233][2108.09420][1109.4603][2311.10790][2209.12473][1503.00021].

Source: https://www.emergentmind.com/topics/gaussian-and-polynomial-kernels