A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance

Abstract: We introduce the Yat kernel $$k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{|\mathbf{x}-\mathbf{w}|^2+\varepsilon},\qquad b\ge 0,\ \varepsilon>0,$$ a rational hidden-unit primitive whose units are Mercer sections over a shared input/weight space. For $b\ge 0$ the kernel is PSD; for $b>0$ it dominates a scaled inverse-multiquadric (IMQ) in the Loewner order, yielding fixed-kernel universality, characteristicness, and strict positive definiteness on every compact domain. The polynomial numerator opens nonradial alignment channels absent from finite IMQ expansions, witnessed by the directional far-field trace $T_\infty g_\varepsilon(\cdot;\mathbf{w},b)(\mathbf{u})=(\mathbf{u}^\top\mathbf{w})^2$. Algebraically, a second finite difference in the bias recovers any IMQ atom from three positive-bias Yat atoms exactly, sharp at three atoms in every dimension at exact pointwise equality. A trained shared-$(b,\varepsilon)$ Yat layer is therefore a finite learned-center expansion in a fixed universal characteristic RKHS, with closed-form norm $\boldsymbolα^\top\mathbf{K}\boldsymbolα$ and explicit diagonal $(|\mathbf{x}|^2+b)^2/\varepsilon$ driving a Rademacher generalization bound.

• The paper introduces the Yat kernel, a novel rational, positive-definite kernel that combines polynomial alignment with IMQ distance to establish universality in RKHS.
• It develops explicit analytic and algebraic identities that enable closed-form computation of RKHS norms and transfer IMQ kernel properties such as characteristicness.
• The kernel’s nonradial alignment channels and directional trace yield exponential efficiency in approximating structured functions, especially in high-dimensional settings.

The Yat Kernel: Polynomial Alignment Meets IMQ Distance in a Universal RKHS

Introduction and Motivation

The paper introduces the Yat kernel, a rational positive-definite kernel defined as

$$k_{b,\varepsilon}(\mathbf{w},\mathbf{x}) = \frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}$$

with $b \geq 0$, $\varepsilon > 0$. This construction provides a hidden-unit primitive for neural architectures whose units are truly kernel sections over the joint input/weight space, in contrast with classical MLP units that are nonlinearities of a linear functional and therefore do not, in general, live in any RKHS.

The Yat kernel splits the gap between radial-basis-function kernels (which provide locality but only radial sensitivity, e.g., Gaussian RBF or inverse-multiquadric (IMQ)) and dot-product kernels (which provide alignment but lack locality), by combining polynomial alignment in the numerator with IMQ distance locality in the denominator. This hybrid structure enables rigorous Hilbert-space control at the hidden-unit level, enabling closed-form norms and generalization bounds unavailable to standard MLPs.

Mercer Structure, Universality, and Characteristicness

The canonical Hilbert norm and generalization properties of kernel expansions require the kernel to be symmetric positive-semidefinite (PSD), i.e., a Mercer kernel. The Yat kernel is proven to be symmetric and PSD for all $b \geq 0$, $\varepsilon > 0$. For $b > 0$, a key result is that $k_{b,\varepsilon}$ dominates a scaled IMQ kernel in the Loewner order:

$b^2 h_\varepsilon \preceq k_{b,\varepsilon}$

allowing the universality and characteristicness of IMQ to be transferred directly to $k_{b,\varepsilon}$. As a consequence, for each fixed $b > 0$, $b \geq 0$0, the RKHS of $b \geq 0$1 is:

• Universal: Dense in $b \geq 0$2 for any compact $b \geq 0$3
• Characteristic: The maximum mean discrepancy metrizes weak convergence of measures
• Strictly positive-definite: Gram matrices at distinct points are strictly positive-definite

Thus, a finite expansion of Yat units with shared $b \geq 0$4 lies in a fixed, universal, strictly-regular RKHS, inheriting the density and metric-embedding properties of the IMQ kernel.

Structural Decomposition and Nonradial Alignment Channels

Unlike classical radial kernels, the Yat kernel's polynomial numerator introduces alignment channels beyond the IMQ's purely radial behavior. A precise channel decomposition shows

$b \geq 0$5

identifying three orthogonal RKHS channels:

• Radial (IMQ): $b \geq 0$6
• Linear-alignment: $b \geq 0$7
• Quadratic-alignment: $b \geq 0$8

A crucial finding is that only the quadratic channel, through its far-field directional trace, does not vanish at infinity. Specifically, for any direction $b \geq 0$9,

$\varepsilon > 0$0

This directional shadow strictly separates Yat functions from those in the IMQ RKHS, which necessarily vanish at infinity, providing structural content not available to any finite IMQ expansion. This nonradial component enables Yat atoms to approximate structured functions (such as quadratic ridge functions) on bounded domains exponentially more efficiently than radial kernels—under appropriate coefficient and center constraints, a single Yat atom can approximate certain quadratics, while IMQ combinations require exponentially many atoms in high dimensions.

Constructive IMQ Embedding via Bias Finite Differences

The analytic proof of universality via Loewner domination is complemented by an exact, algebraic identity: every IMQ atom can be reconstructed exactly by a finite difference of three Yat atoms at distinct positive biases,

$\varepsilon > 0$1

This identity is sharp: the count of three is necessary in every dimension, since no linear combination of fewer than three Yat atoms can match a nontrivial IMQ atom exactly away from the origin. Thus, the Yat span at a single $\varepsilon > 0$2 algebraically contains the IMQ span, and approximation rates for IMQ immediately transfer to Yat at a factor-3 width overhead.

Closed-Form Norm and Generalization Bounds

For finite shared-$\varepsilon > 0$3 Yat expansions, the RKHS norm is given by the Gram form

$\varepsilon > 0$4

where $\varepsilon > 0$5. This closed-form is computable from the expansion coefficients and centers alone, without Lipschitz or spectral surrogates, in sharp contrast to standard MLP layers.

Plugging this norm into the standard RKHS-ball Rademacher framework yields a single-layer generalization bound:

$\varepsilon > 0$6

where the explicit dependence on $\varepsilon > 0$7, $\varepsilon > 0$8, and the radius $\varepsilon > 0$9 is driven by the form of the kernel diagonal, $b \geq 0$0. The bound can be refined empirically by replacing $b \geq 0$1 with the empirical mean of the diagonal elements.

High-dimensional analysis under isotropic Gaussian inputs reveals another key aspect: while radial kernel outputs concentrate to mean zero as $b \geq 0$2, the Yat kernel's dependency on $b \geq 0$3 means it preserves $b \geq 0$4 variance, making it more robust to high-dimensional collapse in practice.

Theoretical and Practical Implications

The Yat kernel's properties have several implications:

• Practical: Shared-parameter Yat layers can be trained as finite kernel expansions with tractable, computable RKHS norms, enabling direct generalization control over hidden-unit capacity without ad hoc regularization or indirect spectral surrogates. The kernel's explicit dependence on alignment and locality allows for finer expressivity in neural architectures, bridging gaps between kernel methods and deep learning.
• Theoretical: The Yat kernel provides the minimal, analytically and algebraically closed primitive unifying alignment (through polynomial features), strict positive-definiteness, universality, and analytic tractability in a single kernel object. The directional trace separation yields lower bounds and atom-count separation between Yat and classical radial methods for structured approximands.
• Depth Composition: While the single-layer theory is exact and fully controlled, composition through depth introduces prefix-dependence: kernel pullbacks via the lower layers preserve RKHS structure conditionally, allowing precise bookkeeping per prefix. However, there is no parameter-independent global Yat-Gram norm for deep stacks; the paper formalizes this as a conjectured impossibility.
• Extensions and Generalizations: The analytic and constructive routes to universality for Yat highlight the potential for extension to other rational or hybrid kernel constructions, potentially enabling more expressive or structurally tailored RKHSs for modern architectures.

Numerical Evidence

Experiments included in the appendices demonstrate:

• Yat layers' tractable closed-form norms correlate with per-class generalization in high-classification scenarios;
• Finite Yat expansions match or outperform standard polynomial or radial baselines, particularly on tasks with alignment structure relative to centers;
• Directional-tail benchmarks quantitatively confirm the theoretical lower bound in atom-count separation between Yat and IMQ/RBF expansions;
• Stacking Yat layers in deep models (e.g., causal LLMs) yields performance on par with classical MLPs at matched parameter counts and compute scales, while maintaining per-layer RKHS observability.

Conclusion

The Yat kernel construction provides a theoretically robust primitive supporting polynomial alignment, IMQ locality, and closed-form norm computation within a strictly universal, characteristic RKHS. It delivers an operational framework for Hilbertian control at the hidden-layer level in neural networks, bridging analytic kernel theory with practical trainable architectures. Its far-field and alignment properties engender new approximation-theoretic and generalization bounds, and its layer-local observability offers a new lens for the study and design of deep neural representations.

This kernel, and the associated analytic and algebraic identities, set the stage for further investigations into trainable kernel units, architectural expressivity, and hybrid kernel-feature frameworks, both in shallow and deep regimes.

Reference: "A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance" (2605.03262).