Separable Operator-Valued Kernel

• Separable operator-valued kernels are matrix-valued positive-definite functions that factor into a scalar kernel and a fixed positive semidefinite matrix, enabling efficient multi-task learning and analysis.
• Their spectral representation via an extension of Bochner's theorem provides tractable analysis and explicit feature maps, achieving scalable kernel approximations using random Fourier features.
• Applications span multi-task learning, structured output predictions, and control theory, where separable kernels simplify inversion and enable robust stability verification.

A separable operator-valued kernel is an operator- or matrix-valued positive-definite function with a specific structural factorization, central to multi-task learning, vector-valued interpolation, the theory of reproducing kernel Hilbert spaces (RKHS), and control theory. The "separable," "decomposable," or sometimes "Mercer-type" form admits efficient representations, enables tractable spectral analysis, and permits scalable algorithmic implementations via explicit feature maps and kernel operators.

1. Definition and Canonical Forms

A shift-invariant $p$-vector-valued kernel $$K: \mathbb{R}^d \times \mathbb{R}^d \rightarrow \mathbb{R}^{p \times p}$$ is separable if there exists a continuous, positive-definite scalar kernel $k_0: \mathbb{R}^d \to \mathbb{R}$ and a fixed $p \times p$ positive semidefinite matrix $C$ such that

$K(x, z) = k_0(x - z)\,C .$

In coordinates, $K_{\ell m}(x, z) = k_0(x - z) \, C_{\ell m}$ (Brault et al., 2016). More generally, on an arbitrary index set $S$ and separable Hilbert space $H$, separability refers to kernels expressible as

$K(s, t) = \Phi(s)\, \Lambda\, \Phi(t)^* ,$

with $\Phi: S \to \mathcal{B}(U, H)$ for some auxiliary Hilbert space $U$ and a fixed positive semidefinite operator $\Lambda \in \mathcal{B}(U)$ (Jorgensen et al., 2024). In finite sums, this can be written as

$K(s,t) = \sum_{i} f_i(s) A_i g_i(t) ,$

with suitable scalar functions and operators $A_i$ (Jorgensen et al., 2024).

2. Operator-valued Bochner Theorem and Spectral Representation

Separable operator-valued kernels inherit a tractable spectral characterization via an extension of Bochner's theorem. For continuous, shift-invariant kernels on $\mathbb{R}^d$, $K(x, z) = K_0(x - z)$ is an operator-valued Mercer kernel if and only if there exists a positive operator-valued measure $M$ such that

$$K(x, z) = \int_{\mathbb{R}^d} e^{-i \langle x - z, \omega \rangle} \, dM(\omega) .$$

For separable $K(x, z) = k_0(x - z)\,C$, this representation specializes to

$$K_0(\delta) = \int_{\mathbb{R}^d} e^{-i \langle \delta, \omega \rangle} \, C\, \hat k_0(\omega) d\omega ,$$

where $\hat k_0$ is the Fourier transform of $k_0$, $C$ is a fixed positive semidefinite matrix, and $d\mu(\omega) = \hat k_0(\omega) \, d\omega$ (Brault et al., 2016, Minh, 2016). Thus, separable OVKs admit a positive-operator-valued spectral density with rank structure entirely governed by $C$.

3. Feature Map Construction and Random Fourier Features

Separable structure yields explicit feature maps and enables scalable kernel approximation. Given $K(x, z) = k_0(x - z) C$ with $C = B B^T$ (Cholesky or spectral factorization), one samples $D$ i.i.d. frequencies $\{\omega_j\}$ from the density $\hat k_0$ and defines the feature map

$$\varphi(x) = \frac{1}{\sqrt{D}} \begin{pmatrix} \cos(\langle x, \omega_1 \rangle) B^T \ \sin(\langle x, \omega_1 \rangle) B^T \ \vdots \ \cos(\langle x, \omega_D \rangle) B^T \ \sin(\langle x, \omega_D \rangle) B^T \end{pmatrix} \in \mathbb{R}^{2D p'}$$

with $p' = \operatorname{rank}(B)$ (Brault et al., 2016, Minh, 2016). The empirical kernel $k_D(x, z) = \varphi(x)^T \varphi(z)$ yields a consistent approximation to $K(x, z)$, inheriting $O_p(1/\sqrt{D})$ convergence rates, up to scaling by $\|C\|_2$, as in classical random Fourier features (Brault et al., 2016).

4. Spectral Decomposition and Mercer-Young Theorem

For continuous kernels $K: X \times X \to \mathbb{R}^{m \times m}$ over a separable metric space $(X, d)$ with probability measure $\mu$ of full support, the generalized Mercer-Young theorem asserts that $K$ is positive definite if and only if the associated integral operator is positive (i.e., for all $f \in L^2(X; \mathbb{R}^m)$, $\iint f(x)^T K(x, y) f(y) d\mu(x) d\mu(y) \geq 0$) (Neuman et al., 2024). The spectral/Hilbert-Schmidt decomposition reads

$$K(x, y) = \sum_{\ell=1}^\infty \lambda_\ell \, \varphi_\ell(x) \, \varphi_\ell(y)^T,$$

for orthonormal vector-valued eigenfunctions and nonnegative eigenvalues, converging uniformly (Neuman et al., 2024). In the separable case, all nontrivial spectral content is inherited from $k_0$ and $C$.

5. Operator Inversion and Control Applications

In Lyapunov analysis of coupled differential-functional equations, separable kernel operators admit explicit inverses. If a block operator $\mathcal{P}$ acts on $$Z = \mathbb{R}^n \times \mathcal{PC}([-r,0], \mathbb{R}^m)$$ and has the separable form $Q(s) = H Z(s)$, $R(s, \theta) = Z(s)^T \Gamma Z(\theta)$, its inverse $\mathcal{Q} = \mathcal{P}^{-1}$ can be written with explicitly parametrized blocks $\hat Q, \hat R, \hat S$ in terms of $P, Q, R, S, H, \Gamma$, and integral projections $K$ (Miao et al., 2017). This algebraic inversion, rather than power-series expansion, yields invariance of essential boundary conditions and enables efficient controller synthesis in polynomial sum-of-squares (SOS) frameworks (Miao et al., 2017).

6. Applications and Algorithmic Implications

Separable OVKs are leveraged in multi-task learning, structured output learning, and vector-valued regression, where modeling inter-task covariance (via $C$) decouples from spatial dependence (via $k_0$). Operator-valued random Fourier features provide order-of-magnitude complexity reductions for large datasets: explicit feature construction replaces kernel matrix inversion with efficient high-dimensional linear models. Empirical results on datasets such as MNIST and synthetic vector field regression demonstrate convergence and runtime benefits for ORFF in the separable setting (Brault et al., 2016).

In systems theory, separable kernels arise as Lyapunov-Krasovskii functionals for coupled time-delay systems, where closed-form algebraic inverses are essential for stability verification and controller synthesis via SOS techniques. The consistent spectral structure of separable kernels guarantees the transfer of convexity between infinite-dimensional and discretized quadratic forms, which is crucial for control and optimization (Miao et al., 2017, Neuman et al., 2024).

7. Structural Characterization and Extension

The dilation-theoretic perspective identifies that every positive-definite operator-valued kernel admits a (possibly infinite-rank) factorization $K(s, t) = V_s^* V_t$; separability is equivalent to the existence of a fixed finite- or countably-generated subspace such that all $V_s^*$ land in its span (Jorgensen et al., 2024). Thus, separable OVKs are precisely those for which the RKHS construction collapses to a classical feature map with fixed output geometry, central in operator-valued kernel theory, stochastic process covariance structures, and dilation/interpolation problems in functional analysis (Jorgensen et al., 2024).