---
title: 'Yat Kernel: Geometric & Neural Computation'
url: https://www.emergentmind.com/topics/yat-kernel
type: topic
---

# Yat Kernel: Geometric & Neural Computation

Searching arXiv for the cited Yat kernel papers to ground the article in the latest preprints.
to=functions.arxiv_search  彩神争霸的 json
{"query":"\"Yat kernel\" OR \"yat-product\" OR \"Neural Matter Networks\" OR \"A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance\" OR \"No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
to=functions.arxiv_search  天天中彩票公众号json
{"query":"ti:\"No More DeLuLu\" OR ti:\"Artificial Neurons That Matter\" OR ti:\"A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
to=functions.arxiv_search ացինjson
{"query":"all:\"2603.12276\" OR all:\"2411.08085\" OR all:\"2605.03262\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"}
The **Yat Kernel** is a rational similarity operator, also called the **Yat-product** or **E-product**, defined on vectors in a common Euclidean space by a squared alignment term in the numerator and a squared-distance term in the denominator. In its stabilized form, it is written
\[
k_{\E}(\mathbf{x},\mathbf{w})=\frac{(\mathbf{x}^\top \mathbf{w})^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon},
\qquad \varepsilon>0,
\]
and in bias-augmented form
\[
k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon},
\qquad b\ge 0,\ \varepsilon>0.
\]
Across recent work, it has been proposed both as a geometric primitive for activation-free neural layers and as a Mercer kernel with a unique reproducing kernel Hilbert space (RKHS), universality on compact domains when \(b>0\), and an explicit connection to inverse-multiquadric (IMQ) kernels [2411.08085] [2603.12276] [2605.03262].

## 1. Definition and geometric content

The core Yat-product combines **quadratic alignment** with **inverse-square proximity**:
\[
\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.
\]
The numerator
\[
A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^2
\]
is large when \(\mathbf{w}\) and \(\mathbf{x}\) are strongly aligned and vanishes when they are orthogonal. Because of the square, the score is **sign-invariant**: aligned and anti-aligned vectors with the same magnitude of inner product yield the same value. The denominator
\[
r(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon
\]
imposes locality, so the response decays as distance grows. Using
\[
\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},
\]
the same operator can be written as
\[
\E(\mathbf{w}, \mathbf{x})=
\frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},
\]
which makes explicit that the denominator depends on both norms and alignment [2603.12276].

A bias-augmented form is used inside Neural Matter Networks:
\[
\mathcal{K}_\E(\mathbf{w}, \mathbf{x}, b)=
\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon}.
\]
A later RKHS treatment standardizes this as
\[
k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=
\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon},
\]
with both arguments in the same ambient space, \(\mathbf{w}\) acting as a trainable center or weight and \(\mathbf{x}\) as the input [2605.03262].

The physics analogy used in the literature is that a Yat unit behaves like a **potential well** centered at \(\mathbf{w}\): strong response requires both alignment and proximity. This differs from a linear neuron, whose response is globally controlled by a hyperplane, and from a purely radial kernel, whose response depends only on distance [2603.12276].

## 2. From pseudo-metric formulation to Mercer kernel

The earliest neural-network formulation frames the operator as one of **Bouhsine’s two products**. For vectors \(e_1,e_2\in\mathbb{R}^n\),
\[
e_1\,E\,e_2=\frac{(e_1\cdot e_2)^2}{\|e_2-e_1\|^2},
\qquad
e_1\,\mathcal E\,e_2=\frac{\|e_2-e_1\|^2}{(e_1\cdot e_2)^2}.
\]
With
\[
d_{ij}=\|e_i-e_j\|^2,\qquad \Omega_{ij}=(e_i\cdot e_j)^2,
\]
that work states that \((\mathbb{R}^n,E)\) is **semi-metric**, while \((\mathbb{R}^n,\mathcal E)\) is **pseudo-metric**. It also emphasizes that the operator is symmetric and non-negative but not associative or distributive, and that it is “not presented as a PSD kernel” or as an inner product of a feature map [2411.08085].

Subsequent work regularizes the denominator by introducing \(\varepsilon>0\) and shifts the mathematical status of the object. A key identity is
\[
\frac{1}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}
=
\int_0^\infty e^{-t(\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon)}\,dt
=
\int_0^\infty e^{-t\varepsilon}e^{-t\|\mathbf{x}-\mathbf{w}\|^2}\,dt.
\]
Hence
\[
k_{\E}(\mathbf{x},\mathbf{w})
=
\int_0^\infty
(\mathbf{x}^\top \mathbf{w})^2
e^{-t\varepsilon}
e^{-t\|\mathbf{x}-\mathbf{w}\|^2}\,dt.
\]
This realizes the Yat kernel as a nonnegative mixture of products of a quadratic polynomial kernel and a Gaussian radial kernel. On every compact set \(K\subset\mathbb{R}^d\), the kernel is then proved symmetric, continuous, and positive definite, and therefore a **Mercer kernel** on \(K\) [2603.12276].

The 2026 RKHS treatment sharpens the parameter dependence. For
\[
k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=
\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon},
\]
the kernel is PSD for every \(b\ge 0\), but the sign condition is stated to be **sharp**: for negative \(b\), the numerator \((\mathbf{x}^\top\mathbf{w}+b)^2\) can be indefinite [2605.03262].

This development resolves an important historical tension. The 2024 neural-network presentation emphasizes pseudo-metric structure and the lack of conventional algebraic properties, whereas the 2026 analyses establish Mercer, RKHS, and universality results for the stabilized kernel under explicit parameter conditions [2411.08085] [2603.12276] [2605.03262].

## 3. RKHS structure, universality, and relation to IMQ kernels

For every compact \(K\subset\mathbb{R}^d\), positive definiteness yields a unique RKHS \(\mathcal H_K\) and feature map \(\phi_K\) such that
\[
k_{\E}(\mathbf{x},\mathbf{y})=
\langle \phi_K(\mathbf{x}),\phi_K(\mathbf{y})\rangle_{\mathcal H_K}.
\]
The corresponding existence-and-uniqueness statement is given explicitly via the Moore–Aronszajn theorem, with uniqueness understood up to isometry [2603.12276].

The later RKHS analysis isolates a decisive distinction between \(b>0\) and \(b=0\). The kernel factors as
\[
k_{b,\varepsilon}(\mathbf{w},\mathbf{x})
=
p_b(\mathbf{x},\mathbf{w})\,h_\varepsilon(\mathbf{x},\mathbf{w}),
\]
where
\[
p_b(\mathbf{x},\mathbf{w})=(\mathbf{x}^\top\mathbf{w}+b)^2,
\qquad
h_\varepsilon(\mathbf{x},\mathbf{w})=
\frac{1}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}.
\]
Thus Yat is the product of a degree-2 polynomial alignment kernel and an IMQ radial kernel. Expanding the numerator gives three channels,
\[
k_{b,\varepsilon}=k_0+k_1+k_2,
\]
with
\[
k_0=b^2 h_\varepsilon,\qquad
k_1=2b(\mathbf{x}^\top\mathbf{w})h_\varepsilon,\qquad
k_2=(\mathbf{x}^\top\mathbf{w})^2 h_\varepsilon.
\]
Accordingly, the Yat RKHS decomposes into a radial IMQ channel, a linear-alignment IMQ channel, and a quadratic-alignment IMQ channel. At \(b=0\), the radial and linear channels vanish; only the quadratic-alignment channel remains [2605.03262].

For \(b>0\), the kernel **Loewner-dominates** a scaled IMQ:
\[
b^2\,h_\varepsilon \preceq k_{b,\varepsilon}.
\]
By Aronszajn’s inclusion theorem,
\[
\mathcal H_{h_\varepsilon}\subseteq \mathcal H_{b,\varepsilon},
\qquad
\|f\|_{\mathcal H_{b,\varepsilon}}
\le \frac{1}{b}\|f\|_{\mathcal H_{h_\varepsilon}}.
\]
Because IMQ is universal on compact subsets of \(\mathbb{R}^d\), the Yat RKHS is also universal when \(b>0\). On compact domains, this yields **characteristicness** and **strict positive definiteness** as well [2605.03262].

A central algebraic result connects Yat atoms to IMQ atoms exactly. With
\[
g_\varepsilon(\cdot;w,b)=\frac{(x^\top w+b)^2}{\|x-w\|^2+\varepsilon},
\]
one has
\[
g_\varepsilon(\cdot;w,3h)-2g_\varepsilon(\cdot;w,2h)+g_\varepsilon(\cdot;w,h)
=
2h^2\,k_{\mathrm{IMQ}}^\varepsilon(\cdot,w).
\]
Thus a second finite difference in the bias recovers any IMQ atom from three positive-bias Yat atoms exactly, and the paper states that this is **sharp at three atoms in every dimension** for exact pointwise equality at generic nonzero centers [2605.03262].

The same work also isolates a nonradial asymptotic channel absent from finite IMQ expansions. For the directional far-field trace,
\[
T_\infty g_\varepsilon(\cdot;w,b)(u)=\lim_{r\to\infty} g_\varepsilon(ru;w,b),
\]
one obtains
\[
T_\infty g_\varepsilon(\cdot;w,b)(u)=(u^\top w)^2,
\]
whereas finite IMQ combinations decay to zero. This separates Yat’s polynomial alignment channels from purely radial IMQ behavior [2605.03262].

## 4. Neural Matter Networks and Yat-based attention

The Yat Kernel is the defining primitive of the **Neural Matter Network** (NMN), in which it replaces the conventional sequence “linear projection + activation + normalization.” For an input \(\mathbf{x}\in\mathbb{R}^d\), an NMN layer with \(n\) units is written
\[
h(\mathbf{x})
=
s\cdot\sum_{i=1}^n \mathcal K_\E(\mathbf{w}_i,\mathbf{x},b_i)
=
s\cdot\sum_{i=1}^n
\frac{(\mathbf{w}_i^\top \mathbf{x}+b_i)^2}{\|\mathbf{w}_i-\mathbf{x}\|^2+\varepsilon},
\]
where the scaling factor is learned via
\[
s=(n/\log(1+n))^\alpha,
\]
with \(\alpha\) learnable. In this design, the Yat-product is the **sole non-linearity** [2603.12276].

An earlier implementation paper gives the layer computation explicitly in Flax/JAX. For inputs \(X\) and weight matrix \(W\), it first computes dot products \(S=XW^\top\), then distances
\[
d_{ij}=\|x_i\|^2+\|w_j\|^2-2S_{ij},
\]
and finally
\[
Y_{ij}=\alpha\cdot\frac{S_{ij}^2}{d_{ij}+\varepsilon}+b_j.
\]
No ReLU, GeLU, or tanh is applied in hidden layers; “all hidden layers are linear-to-yat only” [2411.08085].

The 2026 framework extends the same primitive to transformers. In **Aether-GPT2**, scaled dot-product attention is replaced by Yat-attention:
\[
K_\E(\mathbf{q}_i,\mathbf{k}_j)
=
\frac{(\mathbf{q}_i^\top \mathbf{k}_j)^2}{\|\mathbf{q}_i-\mathbf{k}_j\|_2^2+\varepsilon},
\qquad
\mathbf{S}_{ij}=K_\E(\mathbf{q}_i,\mathbf{k}_j),
\]
followed by
\[
\mathrm{YATAttn}(\mathbf{Q},\mathbf{K},\mathbf{V})
=
\mathrm{softmax}_j(\mathbf{S}_{ij})\,\mathbf{V}.
\]
The MLP block is replaced by **NMN \(d\to 4d\) + Linear \(4d\to d\)**, and the paper states that there is **no LayerNorm in blocks** because the denominator provides intrinsic norm control; attempts to reintroduce LayerNorm caused divergence [2603.12276].

The same paper proves a universal approximation theorem: for compact \(\mathcal X\subset\mathbb{R}^d\), the span of atoms
\[
\frac{(\mathbf{x}\cdot \mathbf{w}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}
\]
is dense in \(C(\mathcal X)\) under the uniform norm. The proof route uses finite-difference identities in \(b\) to express IMQ atoms as linear combinations of Yat atoms, together with the universality of IMQ [2603.12276].

## 5. Regularization, gradients, and representational geometry

A central claim of the Yat literature is that normalization is moved **into the kernel**. Because
\[
\E(\mathbf{w}, \mathbf{x})=
\frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},
\]
the denominator grows like \(\|\mathbf{x}\|^2\) for large \(\|\mathbf{x}\|\), counteracting numerator growth and yielding bounded, scale-aware responses. The 2026 NMN paper describes this as **self-normalization** and **self-regulation**, in contrast with architectures that rely on BatchNorm or LayerNorm [2603.12276].

The same work proves analyticity and a bounded-domain Lipschitz estimate. For fixed \(\varepsilon>0\) and \(\|\mathbf{w}\|_2\le 1\), the map \(\mathbf{x}\mapsto \E(\mathbf{w},\mathbf{x})\) is Lipschitz on the unit ball with constant
\[
L=\frac{2}{\varepsilon}+\frac{4}{\varepsilon^2},
\]
so if \(\|\mathbf{x}'-\mathbf{x}\|\le\delta\),
\[
|\E(\mathbf{w},\mathbf{x}')-\E(\mathbf{w},\mathbf{x})|
\le
\left(\frac{2}{\varepsilon}+\frac{4}{\varepsilon^2}\right)\delta.
\]
It also gives the asymptotic bound
\[
\lim_{k\to\infty}\E(\mathbf{w},k\mathbf{u})
=
\|\mathbf{w}\|^2\cos^2\theta,
\]
showing that the response converges to a finite constant along rays [2603.12276].

The gradient formula is explicit:
\[
\nabla_{\mathbf{e}_i} \E(\mathbf{e}_i,\mathbf{e}_j)
=
\frac{2\langle \mathbf{e}_i, \mathbf{e}_j\rangle}
{\varepsilon + \|\mathbf{e}_i - \mathbf{e}_j\|^2}
\left(
\mathbf{e}_j
-
\frac{\langle \mathbf{e}_i, \mathbf{e}_j\rangle (\mathbf{e}_i - \mathbf{e}_j)}
{\varepsilon + \|\mathbf{e}_i - \mathbf{e}_j\|^2}
\right).
\]
Far from the center,
\[
\lim_{\|\mathbf{x}\|\to\infty}\|\nabla_{\mathbf{x}}\E(\mathbf{w},\mathbf{x})\|=0,
\]
so outliers produce negligible gradients. The paper explicitly describes this as **outlier down-weighting** and reports empirically stable training in **BF16 without gradient clipping** [2603.12276].

An information-geometric interpretation is also developed. Because the kernel induces an RKHS, each representation carries a Hilbertian inner product and norm. The literature emphasizes three geometric features: orthogonality sensitivity, localized potential wells, and an NTK whose entries decay strongly for orthogonal inputs. One paper states that the \(\E\)-NTK eigenvalues decay as \(\mathcal O(k^{-2/d})\), compared with \(\mathcal O(k^{-1/d})\) for the ReLU NTK, and interprets this as yielding smoother function spaces and better-conditioned learning [2603.12276].

For shared-\((b,\varepsilon)\) Yat layers, the RKHS norm is available in closed form. If
\[
f(\cdot)=\sum_{j=1}^m \alpha_j\,k_{b,\varepsilon}(w_j,\cdot),
\]
then
\[
\|f\|_{\mathcal H_{b,\varepsilon}}^2=\boldsymbol{\alpha}^\top \mathbf K\,\boldsymbol{\alpha},
\qquad
K_{ij}=k_{b,\varepsilon}(w_i,w_j).
\]
The diagonal term is
\[
k_{b,\varepsilon}(x,x)=\frac{(\|x\|^2+b)^2}{\varepsilon},
\]
and the corresponding empirical Rademacher complexity bound stated for the RKHS ball \(\mathcal F_{B,b,\varepsilon}\) is
\[
\widehat{\mathrm{Rad}}_n(\mathcal F_{B,b,\varepsilon})
\le
\frac{B(R^2+b)}{\sqrt{n}\sqrt{\varepsilon}}
\]
when \(\|x_i\|\le R\) [2605.03262].

## 6. Empirical results, comparisons, and open questions

On **MNIST**, the 2026 NMN paper studies a 10-neuron prototype classifier with logits
\[
\ell_i(\mathbf{x})=\E(\mathbf{w}_i,\mathbf{x}).
\]
Training uses Adam, learning rate \(0.001\), and 5 epochs. The reported comparison is as follows [2603.12276].

| Model | Test Acc. | \(\Delta\|\mathbf{w}\|\) |
|---|---:|---:|
| Linear | 92.08% | +13.8% |
| Yat | 92.38% | −4.5% |

The same experiment is used to support the claim of **bounded prototype evolution**: linear prototype norms grow, whereas Yat prototype norms shrink slightly. The paper also reports a superposition-style sign-flip test, exploiting the sign invariance of the squared numerator:

| Model | Original Acc. | Acc. after \(\mathbf{w}\to -\mathbf{w}\) |
|---|---:|---:|
| Linear | 92.04% | 0.01% |
| Yat | 92.18% | 87.87% |

The language-modeling results are reported for **GPT-2 “small”** and **Aether-GPT2**, both at roughly \(124\)M parameters, trained on **2.5B tokens from FineWeb** in **BF16** with **AdamW** and the same batch and context length [2603.12276].

| Quantity | GPT-2 | Aether-GPT2 |
|---|---:|---:|
| Final train loss | 4.1969 | 4.0479 |
| Final val loss | 4.6417 | 4.5747 |
| Memory | – | −15–25% |

The paper states a **+1.45%** validation improvement and attributes the memory reduction to dropping LayerNorm activations. It also reports better calibration curves for Aether-GPT2 and stable BF16 training without gradient clipping or LayerNorm [2603.12276].

The earlier 2024 NMN paper reports broader vision benchmarks with Yat-based MLP and ViT variants. On CIFAR-10, CIFAR-100, Caltech101, Oxford Flowers, and STL-10, the summary states that E-neuron models are “consistently better or comparable” to traditional MLP or ViT baselines, while emphasizing that the experiments remain on mid-scale vision tasks and that behavior on massive-scale NLP or structured sequential data is open [2411.08085].

The 2026 RKHS paper adds proof-of-concept experiments outside the original NMN formulation: a single Yat layer on frozen **CLIP ViT-B/32** features for **ImageNet-1k**, and a **262M-parameter causal LM** in which MLP blocks are replaced by Yat blocks. That paper describes these as proof-of-concept results rather than heavily tuned benchmarks and uses them to support the claim that the single-layer RKHS theory is not vacuous [2605.03262].

Several open questions are stated explicitly across the literature. One concerns **depth**: there is “no global, parameter-independent kernel that exactly captures a deep Yat stack’s behaviour,” and a conjectural treatment via pullback kernels is suggested [2605.03262]. Another concerns the choice of \(\varepsilon\); one paper notes an optimal scaling \(\varepsilon^*\propto d\sigma^2\) for noisy inputs, while also presenting practical sensitivity to \(b\) and \(\varepsilon\) [2603.12276] [2605.03262]. Computationally, early analyses estimate an efficiency ratio
\[
\frac{5d-1}{2d+1}\approx 2.5
\]
relative to dot product plus ReLU, whereas a later optimized analysis gives per-neuron counts of \(\sim 4d+4\) for Yat versus \(\sim 2d+1\) for Linear+ReLU, with the same asymptotic \(\Theta(Bnd)\) forward and backward complexity [2411.08085] [2603.12276].

Taken together, these works define the Yat Kernel as a geometrically grounded hidden-unit primitive whose later formulations place it within standard kernel theory without abandoning its original emphasis on alignment, orthogonality, locality, and prototype-like neural computation [2411.08085] [2603.12276] [2605.03262].

Source: https://www.emergentmind.com/topics/yat-kernel