---
title: Kernel Quantile Embeddings
url: https://www.emergentmind.com/topics/kernel-quantile-embeddings-kqes
type: topic
---

# Kernel Quantile Embeddings

Kernel Quantile Embeddings (KQEs) extend the classical approach of embedding probability distributions into reproducing kernel Hilbert spaces (RKHS) by encapsulating generalized quantile information, rather than mere mean structure. This framework generalizes kernel mean embeddings and facilitates the construction of probability metrics with weaker kernel requirements than maximum mean discrepancy (MMD), recovers kernelized forms of sliced Wasserstein distances, and admits computationally efficient estimators with near-linear complexity. KQEs provide a rich, directionally parameterized representation of distributions, demonstrating theoretical guarantees and competitive empirical performance in two-sample testing and related statistical tasks [2505.20433].

## 1. Mathematical Framework and Definition

Given a measurable, continuous, bounded kernel $k\!:\! X\times X \rightarrow \mathbb{R}$ on a Borel space $X$, let $\mathcal{H}$ denote the RKHS with feature map $\psi(x) = k(x,\,\cdot)$ and inner product $\langle \cdot, \cdot \rangle_{\mathcal{H}}$. For a probability measure $P$ on $X$, the classical kernel mean embedding is $\mu_P = \mathbb{E}_{X\sim P} [\psi(X)] \in \mathcal{H}$; $k$ is mean-characteristic if $P \mapsto \mu_P$ is injective.

Quantiles are generalized to RKHS via directional projections. For $u \in S_{\mathcal{H}} = \{ u \in \mathcal{H} : \|u\|_{\mathcal{H}} = 1 \}$, define the pushforward measure $u_{\sharp}P$ on $\mathbb{R}$ by $u(x) = \langle u, \psi(x) \rangle_{\mathcal{H}}$. The one-dimensional $\alpha$-quantile of $u_{\sharp}P$ is denoted $\rho_{u_{\sharp}P}^\alpha$. The kernel quantile embedding (KQE) for $P$, quantile level $\alpha \in [0,1]$, along direction $u \in S_{\mathcal{H}}$, is
\[
\rho_P^{\alpha,u}(\cdot) = \rho_{u_{\sharp}P}^\alpha \; u(\cdot) \in \mathcal{H},
\]
or equivalently $\rho_P^{\alpha,u}(x) = \rho_{u_{\sharp}P}^\alpha \, \langle u, \psi(x) \rangle_{\mathcal{H}}$. The full KQE of $P$ is the collection $Q_k(P) = \{\, \rho_P^{\alpha,u} : \alpha \in [0,1],\, u \in S_{\mathcal{H}}\,\}$ [2505.20433].

## 2. Associated Probability Metrics

KQEs induce a family of statistical distances aggregating quantile discrepancies over directions and quantile levels. Fix a probability measure $\nu$ on $[0,1]$ to weight quantile levels. For two measures $P, Q$ and direction $u \in S_{\mathcal{H}}$, define the $L^p$-quantile difference
\[
\tau_p(P, Q; \nu, u) = \left( \int_0^1 \| \rho_P^{\alpha,u} - \rho_Q^{\alpha,u} \|_\mathcal{H}^p\; \nu(d\alpha) \right)^{1/p}.
\]
To aggregate across $S_{\mathcal{H}}$, define:

- **Expected KQD (e-KQD):**
  \[
  \text{e-KQD}_p(P, Q; \nu, \gamma) = \left( \mathbb{E}_{u \sim \gamma} [\, \tau_p(P, Q; \nu, u)^p ] \right)^{1/p},
  \]
where $\gamma$ is a probability measure on $S_{\mathcal{H}}$.

- **Supremum KQD (sup-KQD):**
  \[
  \text{sup-KQD}_p(P, Q; \nu) = \left( \sup_{u \in S_{\mathcal{H}}} \tau_p(P, Q; \nu, u)^p \right)^{1/p}.
  \]

These KQD metrics subsume kernel mean discrepancies and recover kernelized variants of (max-)sliced Wasserstein distances. Specifically, for $X\subset \mathbb{R}^d$, linear kernel $k(x,y) = x^\top y$, Lebesgue $\nu$, and uniform $\gamma$ on $S^{d-1}$, e-KQD coincides with the sliced Wasserstein distance $SW_p(P, Q)$ and sup-KQD with max-sliced Wasserstein [2505.20433].

## 3. Theoretical Guarantees and Properties

Assume: (A1) $X$ is Hausdorff, separable, and $\sigma$-compact; (A2) $k$ is continuous and separating ($k(x, \cdot)\neq k(y, \cdot)$ for $x\neq y$).

- **Quantile-characteristic Kernels:** If $k$ meets A1 and A2, then the map $P\mapsto Q_k(P)$ is injective (Cramér–Wold in RKHS), so $k$ is quantile-characteristic. Every mean-characteristic kernel is quantile-characteristic, but not conversely.

- **Metric Properties:** With $\nu$ of full support on $[0,1]$, sup-KQD$_p$ defines a metric on $\mathcal{P}_X$. If $\gamma$ also has full support on $S_{\mathcal{H}}$, then e-KQD$_p$ is a metric.

- **Finite-Sample Consistency:** For data $x_{1:n}\sim P$, let $P_n = \frac{1}{n}\sum\delta_{x_i}$. The empirical directional quantile $\rho_{P_n}^{\alpha,u}(x) = [u(x_{1:n})]_{\lceil \alpha n \rceil} \, u(x)$ (with $[\cdot]_j$ denoting the $j$-th order statistic) converges in $\mathcal{H}$-norm to $\rho_P^{\alpha,u}$ at $O_p(n^{-1/2})$ rate, under mild $u_{\sharp}P$ density assumptions. Similarly, empirical e-KQD estimation converges at rate $O(l^{-1/2} + n^{-1/2})$ for $l$ Monte Carlo directions and sample size $n$, under regularity conditions on $\nu$ and kernel moments [2505.20433].

## 4. Algorithmic Implementation and Computational Complexity

**Estimating Directional Quantiles:** Given data $x_{1:n}$, for $u\in S_{\mathcal{H}}$, compute projections $u(x_i) = \langle u,\,\psi(x_i)\rangle_{\mathcal{H}}$, sort, and set $\rho_{P_n}^{\alpha,u}(\cdot) = [u(x_{1:n})]_{\lceil \alpha n \rceil} u(\cdot)$.

**Sampling Directions:** Uniform sampling on $S_{\mathcal{H}}$ is undefined in infinite-dimensional $\mathcal{H}$. Instead, choose a reference measure $\xi$ on $X$, define $C[f](x) = \int k(x,y)f(y)\xi(dy)$, sample $f\sim N(0,C)$, and set $u = f/\|f\|_\mathcal{H}$. A finite-sample proxy uses $z_{1:m}\sim \xi$, empirical covariance $C_m$, and random $\lambda_j \sim N(0,1)$: $f(x) = (1/\sqrt{m})\sum_{j=1}^m \lambda_j k(x,z_j)$.

**e-KQD Estimator Workflow:**
- For $i=1,\ldots,l$ directions: sample $z_{1:m}\sim\xi$, $\lambda_{1:m}\sim N(0,I_m)$; form $f_i$, normalize to $u_i$; sort $u_i(x_{1:n})$, $u_i(y_{1:n})$; compute $\tau_{p,i}^p = \sum_{j=1}^n |[u_i(x)]_j - [u_i(y)]_j|^p f_\nu(j/n)$.
- Return $(1/l \sum_i \tau_{p,i}^p)^{1/p}$.

**Complexity Table:**

| Step                                      | Complexity        | Comment                |
|--------------------------------------------|-------------------|------------------------|
| $f_i(x_{1:n})$, $f_i(y_{1:n})$            | $O(nm)$           | $m$ kernel terms       |
| Norm $\|f_i\|_\mathcal{H}}$ (Gram)        | $O(m^2)$          | $m \times m$ matrix    |
| Sorting                                   | $O(n\log n)$      | Each direction         |
| Total for $l$ directions                  | $O(l\cdot \max\{nm,\,m^2,\,n\log n\})$ |      |

Selecting $l=m=O(\log n)$ yields total cost $O(n\log^2 n)$ (“near-linear”) [2505.20433].

## 5. Empirical Results and Benchmarks

Two-sample testing is performed via permutation thresholding of the test statistic, at significance level $0.05$, comparing e-KQD and sup-KQD to quadratic MMD (U-statistic), linear MMD, MMD–Multi (incomplete U-statistic), sliced Wasserstein, and Sinkhorn.

**Key experimental findings:**
- Power decay under increasing dimension ($d$ up to $512$): e-KQD exhibits the most gradual test power decay, outperforming all fast MMD approximations.
- Laplace vs Gaussian in $\mathbb{R}$, identical first-two moments, polynomial kernel (degree 3): MMD fails as the kernel is not mean-characteristic, but all KQDs succeed, indicating that the kernel is quantile-characteristic.
- Galaxy MNIST ($12288$ dims): e-KQD-centered performs similarly to MMD, while near-linear e-KQD and sup-KQD outperform MMD–Multi.
- CIFAR-10 vs CIFAR-10.1 ($3072$ dims): near-linear KQDs significantly outperform fast MMD approximations at comparable computational cost [2505.20433].

## 6. Connections, Interpretation, and Extensions

KQEs systematically extend kernel mean embeddings by substituting the mean with the comprehensive quantile structure along arbitrary directions in $\mathcal{H}$. Under mild kernel conditions, KQEs uniquely characterize probability distributions, generalizing the role of mean-characteristic kernels. The induced KQD metrics interpolate between kernel mean discrepancy and (max-)sliced Wasserstein distances, providing a flexible continuum of statistical distances for high-dimensional inference. The near-linear time estimators and favorable empirical results suggest practical utility for large-scale hypothesis testing and related statistical machine learning tasks [2505.20433]. A plausible implication is the potential for KQEs to serve as a universal nonparametric tool when classical kernel mean embeddings are insufficient, especially under non mean-characteristic kernels.

Source: https://www.emergentmind.com/topics/kernel-quantile-embeddings-kqes