Reproducing Kernel Banach Spaces

• Reproducing Kernel Banach Spaces are Banach spaces where evaluation at every point is continuous and represented via kernel-induced duality, generalizing RKHS.
• They enable rigorous analysis of non-quadratic regularization and non-Euclidean metrics, broadening the theory and algorithms of kernel methods.
• Canonical constructions like p-norm and Orlicz RKBS support finite kernel expansions and sparsity through representer theorems for efficient learning.

A reproducing kernel Banach space (RKBS) is a Banach space of functions in which point evaluations are continuous and can be represented via a kernel-induced dual pairing, generalizing the structure of reproducing kernel Hilbert spaces (RKHS) to non-Hilbertian geometries. The RKBS paradigm enables the rigorous analysis of non-quadratic regularization, non-Euclidean metrics, structured sparsity, and broadens the theoretical and algorithmic foundations of kernel methods and infinite-width neural networks.

1. Foundational Definition and Characterizations

Let $X$ be a non-empty set. A Banach space $(B,\|\cdot\|_B)$ of real- or complex-valued functions on $X$ is called a reproducing kernel Banach space if, for each $x\in X$, the point-evaluation operator $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$ is continuous, i.e., there exists $C_x < \infty$ so that $|f(x)| \leq C_x \|f\|_B$ for all $f\in B$ (Shin et al., 2024, Lin et al., 2019, Bartolucci et al., 2021, Ikeda et al., 2022).

A fundamental characterization asserts that every RKBS admits a feature space representation: there exists a Banach space $Y$, a feature map $\phi:X \to Y^*$, and a bounded linear operator

$(B,\|\cdot\|_B)$0

with $(B,\|\cdot\|_B)$1 and

$(B,\|\cdot\|_B)$2

(Shin et al., 2024, Bartolucci et al., 2021, Lin et al., 2019). The reproducing (“kernel”) property is then

$(B,\|\cdot\|_B)$3

for any $(B,\|\cdot\|_B)$4 with $(B,\|\cdot\|_B)$5. This generalizes the classical Aronszajn–Moore RKHS construction to the Banach setting by replacing inner product geometry with Banach-space duality and quotient-norm structures.

2. Kernel and Duality Structure

In a generic RKBS setting, a kernel arises as a function $(B,\|\cdot\|_B)$6 such that for a dual pair of Banach spaces $(B,\|\cdot\|_B)$7 and $(B,\|\cdot\|_B)$8, there exists a continuous bilinear form

$(B,\|\cdot\|_B)$9

with reproducing properties

$X$0

(Lin et al., 2019, Ikeda et al., 2022, Heeringa, 27 Mar 2026). Unlike in RKHS, the kernel need not be positive-definite or symmetric; it reflects Banach duality and the structure of the feature space.

A key extension is to vector-valued RKBS (vv-RKBS), in which functions take values in a Banach space $X$1 and the kernel is a map $X$2, where $X$3 denotes the set of twin (bilinear) operators acting across a dual pair $X$4 (Dummer et al., 30 Sep 2025, Zhang et al., 2011, Chen et al., 2019). The reproducing property is then formulated via the duality

$X$5

for all $X$6, $X$7.

3. Canonical Constructions: Feature Space, p-Norm, and Orlicz RKBS

The general framework unifies a diverse range of concrete constructions (Lin et al., 2019, Xu et al., 2014, Song et al., 2011):

• p-norm RKBS: A special case arises when the feature space is $X$8 and the kernel admits a Mercer-type expansion. For a generalized Mercer kernel $X$9 satisfying certain summability conditions, set

$x\in X$0

with dual $x\in X$1. For $x\in X$2 this recovers the classical RKHS. For $x\in X$3 (the ℓ¹ case), strong sparsity properties are present (Xu et al., 2014, Song et al., 2011, Wang et al., 2023).

• Orlicz RKBS: Using Orlicz feature spaces $x\in X$4 defined by a convex Young function $x\in X$5, kernels and norms are built analogously with duals in the conjugate Orlicz space (Lin et al., 2019).
• Integral/Barron–type RKBS: For $x\in X$6 and $x\in X$7 locally compact, and a bounded kernel $x\in X$8, one defines

$x\in X$9

This class includes Barron spaces for infinite-width neural networks (Spek et al., 2022, Bartolucci et al., 2021). For $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$0-norm regularization on $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$1-type feature spaces, an analogous construction yields RKBSs with $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$2-norm margins (Xu et al., 2014, Shin et al., 2024, Kumar et al., 2024).

4. Structural Results: Sums, Decompositions, and Connections to Neural Networks

Given a family $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$3 of RKBSs on the same domain $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$4, their $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$5-direct sum

$$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$6

with infimal convolution norm is itself an RKBS. The feature space is the direct sum of the $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$7 feature spaces, and the kernel is the sum of the $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$8 kernels (Shin et al., 2024).

For integral RKBSs, one obtains a canonical decomposition into an infinite sum of $$\mathrm{ev}_x : B \to \mathbb{R}, \, f \mapsto f(x)$$9-norm RKBSs via measure disintegration: $C_x < \infty$0 where each $C_x < \infty$1 is a $C_x < \infty$2-norm RKBS parametrized over singular measures $C_x < \infty$3 (Shin et al., 2024). This decomposition underlies the analysis of infinite-width (Barron-type) one-layer neural networks, with the solution space structured as an $C_x < \infty$4-sum over $C_x < \infty$5 component spaces.

Moreover, every finite multi-kernel sum, and any hypothesis class formed as a finite sum of such RKBSs, is itself again an RKBS, a key property for multi-block and multi-kernel learning in Banach settings (Shin et al., 2024).

5. Representer Theorems, Sparse Expansions, and Regularization

A critical property for applications is the representer theorem: for a wide class of regularized learning or interpolation schemes posed over an RKBS with kernel $C_x < \infty$6, minimizers are finite kernel expansions: $C_x < \infty$7 for data points $C_x < \infty$8 (Lin et al., 2019, Xu et al., 2014, Kumar et al., 2024, Song et al., 2011). For $C_x < \infty$9-norm and ℓ¹-type RKBS, the dual geometry induces sparsity in the expansion coefficients—sparse representer theorems hold under additional conditions on the kernel and the Banach norm (Wang et al., 2023, Song et al., 2011).

For integral RKBSs and neural network/Barron spaces, regularized empirical risk minimization over the variation norm on the representing measure yields solutions given by atomic (finite) measures, ensuring the network's expressivity aligns with the number of training constraints (Bartolucci et al., 2021, Spek et al., 2022).

6. Optimization: Mirror Descent and Learning Algorithms

Optimization in non-Hilbertian RKBSs naturally exploits the duality and geometry of the Banach space. Mirror descent algorithms, using dual mappings and reproducing kernels, provide efficient optimization techniques and admit precise convergence guarantees. For reflexive, strongly convex Banach spaces endowed with a reproducing kernel, mirror descent achieves linear convergence under strong convexity and smoothness, and $|f(x)| \leq C_x \|f\|_B$0 rates in the constrained setting (Kumar et al., 2024).

The representer property ensures that mirror descent iterates remain within the finite span of the data-induced kernel sections, so even in $|f(x)| \leq C_x \|f\|_B$1-norm and Orlicz RKBSs, infinite-dimensional problems admit finite-dimensional algorithmic reductions.

7. Applications and Structural Insights in Modern Machine Learning

RKBS theory provides the appropriate geometric and functional-analytic framework for learning models that go beyond the scope of RKHS, including:

• Infinite-width neural networks and Barron spaces, admitting norm control via the total variation of representing measures and enabling powerful generalization bounds (Spek et al., 2022, Bartolucci et al., 2021);
• Synthesis of multi-kernel architectures and model classes as infinite or finite sums of RKBSs (Shin et al., 2024);
• Vector-valued hypothesis spaces for multi-task learning, neural operators, and operator-valued regression, built via vv-RKBS frameworks (Dummer et al., 30 Sep 2025, Chen et al., 2019, Zhang et al., 2011);
• Sparse learning and group-lasso regularization, via ℓ¹- and $|f(x)| \leq C_x \|f\|_B$2-norm RKBSs (Wang et al., 2023, Song et al., 2011, Chen et al., 2019);
• Sampling, interpolation, and atomic decompositions in Banach space signal models, including generalizations of Kramer/shannon theorems to RKBSs (Centeno et al., 2018, Christensen, 2010);

Structural decompositions, e.g., the decomposition of integral RKBSs into sums of $|f(x)| \leq C_x \|f\|_B$3-norm spaces, yield fine-grained insight into representational hierarchies underlying neural architectures, enable the design of novel Banach-geometry-based regularizers and provide principled understanding of the complexity, approximation power, and learnability of function classes (Shin et al., 2024, Lu et al., 2024).

• (Shin et al., 2024) Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spaces
• (Spek et al., 2022) Duality for Neural Networks through Reproducing Kernel Banach Spaces
• (Bartolucci et al., 2021) Understanding neural networks with reproducing kernel Banach spaces
• (Xu et al., 2014) Generalized Mercer Kernels and Reproducing Kernel Banach Spaces
• (Song et al., 2011) Reproducing Kernel Banach Spaces with the l1 Norm
• (Song et al., 2011) Reproducing Kernel Banach Spaces with the l1 Norm II: Error Analysis for Regularized Least Square Regression
• (Lin et al., 2019) On Reproducing Kernel Banach Spaces: Generic Definitions and Unified Framework of Constructions
• (Kumar et al., 2024) Mirror Descent on Reproducing Kernel Banach Spaces
• (Centeno et al., 2018) Sampling basis in reproducing kernel Banach spaces
• (Christensen, 2010) Sampling in reproducing kernel Banach spaces on Lie groups
• (Dummer et al., 30 Sep 2025) Vector-Valued Reproducing Kernel Banach Spaces for Neural Networks and Operators
• (Chen et al., 2019) Vector-valued Reproducing Kernel Banach Spaces with Group Lasso Norms
• (Heeringa, 27 Mar 2026) Characterization of the reproducing structure of the Bessel potential spaces beyond $|f(x)| \leq C_x \|f\|_B$4
• (Lu et al., 2024) Which Spaces can be Embedded in Lp-type Reproducing Kernel Banach Space? A Characterization via Metric Entropy
• (Wang et al., 2023) Sparse Representer Theorems for Learning in Reproducing Kernel Banach Spaces
• (Owhadi et al., 2015) Separability of reproducing kernel spaces
• (Higuera et al., 6 Feb 2026) Featured Reproducing Kernel Banach Spaces for Learning and Neural Networks
• (Ikeda et al., 2022) Koopman and Perron-Frobenius Operators on reproducing kernel Banach spaces
• (Zhang et al., 2011) Vector-valued Reproducing Kernel Banach Spaces with Applications to Multi-task Learning