Relative Reproducing Kernels

• Relative reproducing kernels are generalizations of classical kernels that encode differences between function values using operator constructions in Hilbert and Banach spaces.
• They enable precise interpolation, sampling, and regularization in multi-task learning, graph signal processing, and stochastic processes by leveraging unique cocycle properties.
• Their operator-theoretic framework, including inclusion relations and feature map characterizations, provides essential insights for developing robust learning algorithms.

A relative reproducing kernel is a generalization of the classical reproducing kernel concept in Hilbert and Banach spaces of functions. Rather than encoding the reproduction of pointwise function values, relative reproducing kernels encode the reproduction of differences or, more generally, relative measurements between function values at pairs of points. This structure is central in several analytic frameworks including difference-penalized learning, multi-task and vector-valued extensions, operator-theoretic sampling, and stochastic process modeling, where the focus is on relations between function values (e.g., increments, conditional mean differences) rather than their absolute magnitudes (Ebadian et al., 2016, Jorgensen et al., 2016, Zhang et al., 2011, Jorgensen et al., 2017).

1. Definitions and Formal Structure

Let $X$ be a nonempty set and $\mathcal{Y}$ a normed space. A $\mathcal{Y}$-reproducing kernel is a function $K: X \times X \to \mathcal{L}(\mathcal{Y})$ satisfying the standard positive-definiteness condition

$$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$

for all $x_i \in X$ and $y_i \in \mathcal{Y}$ ($1 \leq i \leq n$). The associated vector-valued RKHS $\mathcal{H}_K$ is characterized by the feature operator $K_x y := K(\cdot,x)y$ and the reproducing identity $\mathcal{Y}$0 (Ebadian et al., 2016).

A relative reproducing kernel Hilbert space (RRKHS) is a vector-valued RKHS $\mathcal{Y}$1 for which there exists a mapping

$\mathcal{Y}$2

such that for every $\mathcal{Y}$3,

$\mathcal{Y}$4

For scalar-valued RKHS, the canonical construction is $\mathcal{Y}$5. The feature $\mathcal{Y}$6 thus represents the difference (or cocycle) structure of the function space (Ebadian et al., 2016, Jorgensen et al., 2016).

The concept extends further to Banach spaces (RRKBS) via semi-inner products and duality mappings, where uniqueness of representation typically requires reflexivity and uniform convexity (Ebadian et al., 2016).

2. Main Theorems and Operator Properties

In the Hilbert space setting, the relative kernel operator $\mathcal{Y}$7 possesses several crucial properties:

• Uniqueness: $\mathcal{Y}$8 is unique for each pair $\mathcal{Y}$9, a consequence of the Riesz representation theorem.
• Additivity (Cocycle): The operators satisfy

$\mathcal{Y}$0

reflecting a cocycle or difference-chain property pivotal in interpolation and potential theory.

• Continuous Inclusions: For a Hilbert space $\mathcal{Y}$1 with both an absolute kernel $\mathcal{Y}$2 (standard RKHS) and relative kernel $\mathcal{Y}$3 (RRKHS), the subspace inclusions hold:

$\mathcal{Y}$4

Every feature generated by $\mathcal{Y}$5 can be expressed as a linear combination of features from $\mathcal{Y}$6 (Ebadian et al., 2016).

In the Banach setting, the difference evaluation operator $\mathcal{Y}$7 is required to be bounded. The representing map via the duality $\mathcal{Y}$8 yields $\mathcal{Y}$9 as a bounded operator from $K: X \times X \to \mathcal{L}(\mathcal{Y})$0 to $K: X \times X \to \mathcal{L}(\mathcal{Y})$1 (Ebadian et al., 2016).

3. Relative Reproducing Kernels and Inclusion Relations

Relative reproducing kernels are deeply related to inclusion relations between RKHSs. For kernels $K: X \times X \to \mathcal{L}(\mathcal{Y})$2 and $K: X \times X \to \mathcal{L}(\mathcal{Y})$3 on $K: X \times X \to \mathcal{L}(\mathcal{Y})$4, $K: X \times X \to \mathcal{L}(\mathcal{Y})$5 if and only if $K: X \times X \to \mathcal{L}(\mathcal{Y})$6 is positive-definite, which is equivalent to $K: X \times X \to \mathcal{L}(\mathcal{Y})$7 with bounded inclusion. In this case, $K: X \times X \to \mathcal{L}(\mathcal{Y})$8 is called relative to $K: X \times X \to \mathcal{L}(\mathcal{Y})$9 (Zhang et al., 2011).

The inclusion constant can be characterized as

$$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$0

which coincides with the operator norm of the embedding. The inclusion is with equivalent norms $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$1 if there exist bounds $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$2 such that

$$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$3

for all $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$4.

Feature-map characterizations show that $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$5 if and only if there exists a bounded linear operator $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$6 between the feature spaces such that $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$7. This operator-theoretic perspective unifies inclusions, relative kernel constructions, and provides comparison criteria for spectral measures, particularly for translation-invariant kernels and Hilbert-Schmidt expansions.

4. Operator-Theoretic and Sampling Framework

A comprehensive operator-theoretic analysis shows that, given an RKHS $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$8 with kernel $$\sum_{i,j=1}^n \left\langle K(x_i, x_j) y_j, y_i \right\rangle_{\mathcal{Y}} \geq 0$$9, the “sampling operator” $x_i \in X$0 and its adjoint $x_i \in X$1 produce a canonical factorization: $x_i \in X$2 where $x_i \in X$3 is a relative reproducing kernel on $x_i \in X$4, and $x_i \in X$5 is the Dirac evaluation at $x_i \in X$6 provided $x_i \in X$7. This identification leads to closed-form Gram-matrix and inverse formulas for the relative kernel in sampling, network, and interpolation problems (Jorgensen et al., 2016).

Necessary and sufficient conditions for $x_i \in X$8 are given by boundedness of $x_i \in X$9, in terms of finite Gram matrices $y_i \in \mathcal{Y}$0, providing explicit links between the geometry of RKHS, sampling theory, and the structure of relative reproducing kernels.

5. Concrete Examples

• Scalar-valued (classical) RRKHS: For real-valued RKHS with kernel $y_i \in \mathcal{Y}$1, $y_i \in \mathcal{Y}$2 satisfies $y_i \in \mathcal{Y}$3 (Ebadian et al., 2016).
• Vector-valued multi-task kernels: With $y_i \in \mathcal{Y}$4 and $y_i \in \mathcal{Y}$5, $y_i \in \mathcal{Y}$6 generates the corresponding RRKHS; when $y_i \in \mathcal{Y}$7, this reduces to a direct sum of scalar-valued cases (Ebadian et al., 2016).
• Graph Laplacians and resistive networks: In the energy Hilbert space of a graph, the difference $y_i \in \mathcal{Y}$8 is reproduced by a dipole feature $y_i \in \mathcal{Y}$9, with the effective resistance metric given by $1 \leq i \leq n$0, and the induced relative kernel being the Laplacian matrix (Jorgensen et al., 2016).
• Gaussian processes: In the Cameron–Martin space of Brownian motion, restriction to discrete subsets yields relative kernels that are inverses of Gram matrices of the Brownian kernel, with explicit closed forms for norms of $1 \leq i \leq n$1 and the relative matrices (Jorgensen et al., 2016).

6. Applications and Open Questions

Relative reproducing kernels are fundamental in regularization and interpolation strategies where differences between function values are more natural than absolute values—multi-task learning, difference-penalized estimators, and graph-based signal processing.

The RRKHS and RRKBS structures provide the mathematical framework for representing solutions to variational problems with difference-based regularizers and have direct implications in vector-valued and structured data settings. For learning theory, representer theorems for Banach or Hilbert settings with relative regularizers depend crucially on the properties of the relative kernel.

Open questions include:

• Characterizing minimal conditions, beyond reflexivity and uniform convexity, for the uniqueness of representing elements in RRKBS.
• Establishing representer theorems for relative regularization risks in Banach spaces, i.e., demonstrating that minimizers have sparse representations in terms of $1 \leq i \leq n$2.
• Studying stability and generalization properties for algorithms based on relative kernels, including empirical risk bounds (Ebadian et al., 2016).

7. Significance and Perspectives

Relative reproducing kernels unify several perspectives across operator theory, spectral analysis, learning theory, and stochastic processes. Their identification with difference operators (cocycles) elucidates inclusion/exclusion relationships between kernels, structure of feasible function classes in learning, and connections to physical models (e.g., networks, diffusions). The operator and sampling-theoretic framework developed for relative kernels provides a toolkit for analyzing and constructing function spaces where the focus is on relations or increments—critical in modern applications across statistics, machine learning, and applied mathematics (Ebadian et al., 2016, Jorgensen et al., 2016, Zhang et al., 2011, Jorgensen et al., 2017).