Double-Space Tensor-Product RKHS Framework

• Double-Space Tensor-Product RKHS is a framework that organizes learning over coupled Hilbert spaces using additive (direct-sum) and multiplicative (tensor-product) constructions.
• It distinguishes between modeling superposed components (e.g., linear vs. nonlinear, high- vs. low-frequency) through separate RKHSs and their unique decompositions.
• Adaptive algorithms like CHYPASS exploit component-specific dictionary updates and hyperplane projections to optimize performance and manage model complexity.

Searching arXiv for the cited papers to ground the article in the referenced literature. The Double-Space Tensor-Product RKHS framework denotes a family of constructions in which learning is organized over two coupled Hilbert-space structures: either a Cartesian product or sum of reproducing kernel Hilbert spaces for additive multicomponent function estimation, or a tensor-product RKHS for multiplicative interactions on product domains. In the formulation of “Adaptive Learning in Cartesian Product of Reproducing Kernel Hilbert Spaces” (Yukawa, 2014), the central object is the Cartesian product $H^\times = H_1 \times H_2$ together with the sum space $H^+ = H_1 + H_2$, used to estimate or track nonlinear functions composed of multiple components such as “linear and nonlinear components” or “high- and low- frequency components.” The broader literature represented here connects that additive double-space viewpoint to tensor-product RKHS constructions used for hybrid tensor decomposition (Larsen et al., 2024), correspondence-free bundle adjustment over geometry and semantics (Zhang et al., 2024), joint distribution learning (Filipovic et al., 2021), algebraic tensor closure properties for RKHSs (Giannakis et al., 2024), dynamic multilayer network modeling (Tang et al., 5 Sep 2025), Gaussian/Hermite infinite tensor products (Gnewuch et al., 2021), and multilinear spectral regularization in tensor-product RKHSs (Signoretto et al., 2013). Taken together, these works establish a technical distinction between additive superposition in $H_1 \times H_2$ or $H_1+H_2$, and multiplicative interaction modeling in $H_1 \otimes H_2$, while also suggesting a common design pattern: define mode-specific RKHSs, choose additive or multiplicative coupling according to the target structure, and exploit representer-based finite reductions for optimization.

1. Formal double-space constructions

In the additive framework of (Yukawa, 2014), two RKHSs over a common input space $U$, $(H_1,\langle\cdot,\cdot\rangle_{H_1})$ and $(H_2,\langle\cdot,\cdot\rangle_{H_2})$ with kernels $k_1$ and $k_2$, generate the Cartesian product Hilbert space

$H^+ = H_1 + H_2$0

Its inner product and norm are

$H^+ = H_1 + H_2$1

$H^+ = H_1 + H_2$2

The associated sum space is

$H^+ = H_1 + H_2$3

with RKHS norm defined by the minimal decomposition: $H^+ = H_1 + H_2$4 When $H^+ = H_1 + H_2$5, the decomposition is unique, $H^+ = H_1 + H_2$6, and the map

$H^+ = H_1 + H_2$7

is a Hilbert-space isomorphism (Yukawa, 2014).

The reproducing kernel of $H^+ = H_1 + H_2$8 is the additive kernel

$H^+ = H_1 + H_2$9

and with positive weights $H_1 \times H_2$0 one has the weighted sum-space kernel

$H_1 \times H_2$1

with weighted inner product satisfying

$H_1 \times H_2$2

(Yukawa, 2014).

By contrast, the tensor-product side uses the multiplicative kernel

$H_1 \times H_2$3

which is the reproducing kernel of the tensor-product RKHS $H_1 \times H_2$4 on the product domain $H_1 \times H_2$5 or on heterogeneous product domains. In (Yukawa, 2014), this construction is explicitly distinguished from the additive sum-space formalism and is “not directly developed in the paper,” though it is identified as “the standard way to model multiplicative interactions.”

Several later works instantiate the tensor-product viewpoint explicitly. In CP-HiFi, the ambient product space is

$H_1 \times H_2$6

where the $H_1 \times H_2$7 are RKHSs for continuous modes and the Euclidean factors represent discrete modes (Larsen et al., 2024). In RKHS-BA, the “Double-Space domain” is $H_1 \times H_2$8 for geometry and semantics, with a separable kernel

$H_1 \times H_2$9

on the tensor-product RKHS $H_1+H_2$0 (Zhang et al., 2024). In adaptive joint distribution learning, the hypothesis space is $H_1+H_2$1 with product kernel

$H_1+H_2$2

(Filipovic et al., 2021). These constructions share the product-kernel mechanism, but they differ in objective, parameterization, and interpretation.

2. Additive double-space modeling in Cartesian-product and sum-space RKHSs

The additive formulation of (Yukawa, 2014) is tailored to multicomponent targets of the form

$H_1+H_2$3

Each component admits its own kernel expansion,

$H_1+H_2$4

and in the adaptive setting each RKHS maintains its own dictionary $H_1+H_2$5, $H_1+H_2$6, generating subspaces

$H_1+H_2$7

with dictionary sum-space $H_1+H_2$8 (Yukawa, 2014).

This componentwise organization is particularly natural when different kernels capture different structures. The data synthesis explicitly lists “linear and nonlinear components” and “high- and low- frequency components” as representative targets (Yukawa, 2014). The structural examples include the linear kernel

$H_1+H_2$9

and the Gaussian kernel

$H_1 \otimes H_2$0

A key result attributed to Minh (2010) states that polynomial, including linear, RKHS and Gaussian RKHS intersect trivially: $H_1 \otimes H_2$1 on sets $H_1 \otimes H_2$2 with nonempty interior. Hence

$H_1 \otimes H_2$3

so the $H_1 \otimes H_2$4–$H_1 \otimes H_2$5 isomorphism holds in the linear-plus-Gaussian case (Yukawa, 2014).

By contrast, for two Gaussian RKHSs with $H_1 \otimes H_2$6, the spaces are nested: $H_1 \otimes H_2$7 and the norms satisfy

$H_1 \otimes H_2$8

The synthesis explicitly notes that this is “not a direct sum in that case,” so the Cartesian-product formulation is preferred (Yukawa, 2014).

The sum-space viewpoint also supports a batch regularization interpretation. Kernel ridge regression in the additive space takes the form

$H_1 \otimes H_2$9

with representer

$U$0

where $U$1 is the Gram matrix of $U$2. Componentwise regularization,

$U$3

is stated to be equivalent to weighted sum-space regularization via the weighted norm construction (Yukawa, 2014).

A plausible implication is that the additive double-space formalism supplies an RKHS-native decomposition language for heterogeneous priors without collapsing all structure into a single kernel. In this reading, the direct-sum case provides not only algebraic simplification but also interpretability, because the decomposition $U$4 is unique.

3. Iterative projection learning: HYPASS and CHYPASS

The adaptive algorithm in (Yukawa, 2014) combines multikernel adaptive filtering with the algorithm of hyperplane projection along affine subspace. For a new sample $U$5, the zero-instantaneous-error hyperplane restricted to the current dictionary sum-space is

$U$6

The relaxed projection update is

$U$7

where $U$8 denotes orthogonal projection in $U$9 (Yukawa, 2014).

For a general hyperplane $(H_1,\langle\cdot,\cdot\rangle_{H_1})$0,

$(H_1,\langle\cdot,\cdot\rangle_{H_1})$1

In the direct-sum case $(H_1,\langle\cdot,\cdot\rangle_{H_1})$2, the projection onto $(H_1,\langle\cdot,\cdot\rangle_{H_1})$3 decomposes componentwise: $(H_1,\langle\cdot,\cdot\rangle_{H_1})$4 Each component projection is computed from normal equations. If

$(H_1,\langle\cdot,\cdot\rangle_{H_1})$5

then

$(H_1,\langle\cdot,\cdot\rangle_{H_1})$6

where

$(H_1,\langle\cdot,\cdot\rangle_{H_1})$7

In particular,

$(H_1,\langle\cdot,\cdot\rangle_{H_1})$8

(Yukawa, 2014).

The key general-case update is the Cartesian HYPASS, or CHYPASS, formula: $(H_1,\langle\cdot,\cdot\rangle_{H_1})$9 This form “does not require the direct-sum assumption” and enforces

$(H_2,\langle\cdot,\cdot\rangle_{H_2})$0

by projecting along the sum of the componentwise projected kernel sections (Yukawa, 2014).

Dictionary maintenance is kernel-specific. For the linear kernel, “a fixed orthonormal basis can be used,” whereas for the Gaussian kernel the dictionary can grow online via coherence criteria, with pruning by shrinkage or $(H_2,\langle\cdot,\cdot\rangle_{H_2})$1. To reduce complexity, one may use selected subsets $(H_2,\langle\cdot,\cdot\rangle_{H_2})$2, for example the $(H_2,\langle\cdot,\cdot\rangle_{H_2})$3 nearest neighbors to $(H_2,\langle\cdot,\cdot\rangle_{H_2})$4 by Gaussian coherence, replacing $(H_2,\langle\cdot,\cdot\rangle_{H_2})$5 by $(H_2,\langle\cdot,\cdot\rangle_{H_2})$6 in the update (Yukawa, 2014).

The paper synthesis states that each $(H_2,\langle\cdot,\cdot\rangle_{H_2})$7 is a closed convex hyperplane and that the relaxed projection with $(H_2,\langle\cdot,\cdot\rangle_{H_2})$8 is standard for Fejér-type monotonicity in projection methods, while also noting that “the paper does not present a formal convergence theorem” (Yukawa, 2014). This is important because the mechanism is explicitly characterized by iterative orthogonal projections, but the article’s evidence is numerical rather than theorem-based.

The stated per-iteration complexities are:

Case	Complexity
Linear-Gaussian	CHYPASS $(H_2,\langle\cdot,\cdot\rangle_{H_2})$9
Two-Gaussian	CHYPASS $k_1$0, $k_1$1

The $k_1$2 term is attributed to inversion of the small selected Gram submatrix (Yukawa, 2014).

A one-iteration worked example is given for a linear RKHS plus a normalized Gaussian RKHS. With selected subsets $k_1$3 and $k_1$4, one computes

$k_1$5

$k_1$6

where $k_1$7 solves $k_1$8. The denominator is

$k_1$9

and because $k_2$0 in RKHSs,

$k_2$1

The update becomes

$k_2$2

$k_2$3

or componentwise

$k_2$4

(Yukawa, 2014).

4. From additive double-space to tensor-product RKHSs

The synthesis accompanying (Yukawa, 2014) makes the additive-versus-multiplicative distinction explicit. Additive modeling uses $k_2$5 or $k_2$6 when the target decomposes as

$k_2$7

with component kernels combined through

$k_2$8

Multiplicative modeling instead uses the tensor-product RKHS $k_2$9, typically on product domains, with

$H^+ = H_1 + H_2$00

The text states that this tensor-product construction “is different from the additive sum-space” and that the paper “does not implement tensor-product RKHS updates,” but also states that “the HYPASS projection principle extends” formally to the tensor-product space (Yukawa, 2014).

That extension is summarized as follows: in $H^+ = H_1 + H_2$01, using kernel section $H^+ = H_1 + H_2$02 and a dictionary of product atoms $H^+ = H_1 + H_2$03, the projection retains the form

$H^+ = H_1 + H_2$04

where $H^+ = H_1 + H_2$05 is a subspace of $H^+ = H_1 + H_2$06 (Yukawa, 2014). The text immediately adds that “complexity grows with the number of product atoms, so sparsification becomes even more critical.”

This distinction reappears in later tensor-product works, but there the multiplicative form is not an extension remark; it is the primary modeling object. In CP-HiFi, a rank-$H^+ = H_1 + H_2$07 mixed continuous/discrete tensor is modeled as

$H^+ = H_1 + H_2$08

and evaluation at mixed coordinates factorizes as a product over continuous and discrete modes (Larsen et al., 2024). In the dynamic multilayer network setting, the “double-space tensor-product RKHS” appears when a layer-time varying core $H^+ = H_1 + H_2$09 is placed in

$H^+ = H_1 + H_2$10

with kernel

$H^+ = H_1 + H_2$11

leading to expansions

$H^+ = H_1 + H_2$12

and norms

$H^+ = H_1 + H_2$13

(Tang et al., 5 Sep 2025). In RKHS-BA, geometry and semantics are modeled on $H^+ = H_1 + H_2$14 with product kernel $H^+ = H_1 + H_2$15, and transformed frame similarity is computed through inner products in $H^+ = H_1 + H_2$16 (Zhang et al., 2024). In JDL, the tensor-product space $H^+ = H_1 + H_2$17 is paired with empirical $H^+ = H_1 + H_2$18 spaces to form what the paper calls a double orthogonality structure across two spaces per marginal (Filipovic et al., 2021).

This suggests a useful editorial distinction. The phrase “double-space” is used in at least two closely related ways in the supplied literature: first, for additive two-component RKHS decompositions in a Cartesian product or sum space (Yukawa, 2014); second, for product-domain constructions where two or more spaces are coupled multiplicatively through a tensor-product RKHS (Zhang et al., 2024, Larsen et al., 2024, Filipovic et al., 2021, Tang et al., 5 Sep 2025). The data support both usages, but they are not interchangeable.

5. Representative instantiations across the literature

Several works in the supplied corpus realize double-space or tensor-product RKHS constructions in distinct application regimes. The table summarizes only formulations explicitly stated in the data.

Work	Space construction	Kernel form
Adaptive learning (Yukawa, 2014)	$H^+ = H_1 + H_2$19, $H^+ = H_1 + H_2$20	$H^+ = H_1 + H_2$21
CP-HiFi (Larsen et al., 2024)	$H^+ = H_1 + H_2$22	modewise RKHS kernels $H^+ = H_1 + H_2$23
RKHS-BA (Zhang et al., 2024)	$H^+ = H_1 + H_2$24 on $H^+ = H_1 + H_2$25	$H^+ = H_1 + H_2$26
JDL (Filipovic et al., 2021)	$H^+ = H_1 + H_2$27	$H^+ = H_1 + H_2$28
Dynamic multilayer networks (Tang et al., 5 Sep 2025)	$H^+ = H_1 + H_2$29	$H^+ = H_1 + H_2$30
Gaussian/Hermite tensor products (Gnewuch et al., 2021)	countable tensor products of RKHSs	product kernels $H^+ = H_1 + H_2$31 and $H^+ = H_1 + H_2$32
TP-RKHS with spectral penalties (Signoretto et al., 2013)	$H^+ = H_1 + H_2$33	$H^+ = H_1 + H_2$34

In CP-HiFi, the continuous-mode factors obey representer expansions on the union of observed coordinates: $H^+ = H_1 + H_2$35 and the evaluated factor matrix becomes

$H^+ = H_1 + H_2$36

(Larsen et al., 2024). The method is motivated by the statement that CP-HiFi “does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data” (Larsen et al., 2024).

In RKHS-BA, continuous landmark functions are represented either in a vector-valued RKHS or as scalar functions on $H^+ = H_1 + H_2$37: $H^+ = H_1 + H_2$38 and the pairwise alignment energy is

$H^+ = H_1 + H_2$39

The cross-term expands as

$H^+ = H_1 + H_2$40

which the exposition identifies as the tensor-product scalar counterpart of the paper’s vector-valued semantic weighting (Zhang et al., 2024).

In JDL, the product RKHS is combined with empirical $H^+ = H_1 + H_2$41 and $H^+ = H_1 + H_2$42. The paper describes the computational device as working “simultaneously in two inner-product spaces”: the RKHS inner products and the empirical $H^+ = H_1 + H_2$43 inner products, yielding an adaptive basis that is orthonormal in the RKHS and diagonal in empirical $H^+ = H_1 + H_2$44 (Filipovic et al., 2021). This is a notably different meaning of double-space than the Cartesian $H^+ = H_1 + H_2$45 construction, but it still depends on a two-space coupling principle.

The algebraic paper on reproducing kernel Hilbert algebras proves that the class of RKHAs is closed under Hilbert space tensor product and pullback, with tensor-product kernel

$H^+ = H_1 + H_2$46

and multiplication bound controlled by the product of factor bounds (Giannakis et al., 2024). That result is structural rather than algorithmic, but it situates product-RKHS construction within a broader categorical and algebraic framework.

The paper on countable tensor products of Hermite spaces and spaces of Gaussian kernels proves that, under square-summable Gaussian shape parameters, the tensor-product Gaussian space is isometrically isomorphic to a tensor-product Hermite space, with an explicit isometry that “respects point evaluations” and is also an $H^+ = H_1 + H_2$47-isometry (Gnewuch et al., 2021). This provides an infinite-dimensional realization of tensor-product RKHS equivalences.

Finally, the multilinear spectral-penalty work formulates learning in TP-RKHSs over Cartesian products $H^+ = H_1 + H_2$48 with kernel

$H^+ = H_1 + H_2$49

and the data block identifies “a novel representer theorem suitable for existing as well as new spectral penalties for tensors” (Signoretto et al., 2013). The synthesis supplied with the prompt marks this presentation as paraphrastic rather than quoted, so its detailed derivations should be read as a standard-form exposition consistent with the paper rather than verbatim reconstruction.

6. Empirical behavior, practical criteria, and interpretive cautions

For the additive CHYPASS framework, the supplied experiments are concrete. In the linear-plus-Gaussian experiment,

$H^+ = H_1 + H_2$50

“CHYPASS and MKNLMS (two kernels) outperform single-kernel KNLMS/HYPASS, with smaller dictionaries.” The reported mean dictionary sizes are KNLMS 31.9, HYPASS 31.6, MKNLMS 17.8, and CHYPASS 17.7 (Yukawa, 2014). In the two-Gaussian experiment,

$H^+ = H_1 + H_2$51

with $H^+ = H_1 + H_2$52, $H^+ = H_1 + H_2$53, and $H^+ = H_1 + H_2$54, “CHYPASS achieves better MSE than MKNLMS, with smaller or comparable dictionaries,” with mean sizes KNLMS 205.0, HYPASS 205.8, MKNLMS 147.6, CHYPASS 149.3 (Yukawa, 2014). Similar behavior is reported for a partially linear dynamic system and for real laser and CO$H^+ = H_1 + H_2$55 data (Yukawa, 2014).

In CP-HiFi, the empirical outcomes listed in the data concern sampling alignment rather than dictionary size. The paper reports that with dense aligned sampling, CP and CP-HiFi are similar; with fewer aligned points, CP appears jaggier while CP-HiFi preserves smoothness; a single misaligned point can induce spurious behavior in CP’s continuous-mode factor while CP-HiFi remains well-behaved; and with fully unaligned sampling across fibers, CP-HiFi best matches ground truth (Larsen et al., 2024). In RKHS-BA, semantic information is reported to improve robustness and running time, with cited examples including “mean ATE 0.584 m with semantics vs. 0.664 m color-only” on TartanAir and lower translational and rotational drifts on SemanticKITTI (Zhang et al., 2024). In JDL, scalability “up to n ≈ 10^7” is explicitly reported, with positivity and normalization enforced by construction and conditional distributions obtained by normalization of the learned nonnegative joint table (Filipovic et al., 2021).

These results support a practical division of labor. The additive double-space machinery of (Yukawa, 2014) is directed at online estimation/tracking of superposed components and emphasizes selective dictionary updates, hyperplane projections, and decomposition structure. The multiplicative tensor-product formulations emphasize product-domain interactions, representer-based finite reductions, and mode-coupled optimization.

Three cautions follow directly from the data.

First, additive and multiplicative constructions should not be conflated. The prompt material repeatedly separates $H^+ = H_1 + H_2$56 from $H^+ = H_1 + H_2$57 (Yukawa, 2014). Second, the direct-sum isomorphism $H^+ = H_1 + H_2$58 requires $H^+ = H_1 + H_2$59; this fails for nested RKHS pairs such as two Gaussians of different bandwidths (Yukawa, 2014). Third, not every “double-space” construction refers to the same geometry. In JDL, “double-space” concerns simultaneous RKHS and empirical $H^+ = H_1 + H_2$60 orthogonality (Filipovic et al., 2021), whereas in RKHS-BA it refers to geometry–semantics coupling on a product domain (Zhang et al., 2024).

A plausible implication is that “Double-Space Tensor-Product RKHS framework” functions less as a single canonical formalism than as a recurring architectural pattern in RKHS research. In one branch, it denotes additive superposition across multiple RKHS components, as in CHYPASS (Yukawa, 2014). In another, it denotes multiplicative coupling across domains or modes through a product kernel, as in CP-HiFi (Larsen et al., 2024), RKHS-BA (Zhang et al., 2024), JDL (Filipovic et al., 2021), and related tensor-product constructions [(Tang et al., 5 Sep 2025); (Gnewuch et al., 2021); (Signoretto et al., 2013)]. The common thread is the deliberate separation of structure across spaces, together with explicit control of how those spaces are recombined—by direct sum, Cartesian product, or tensor product—according to the target phenomenon.