Papers
Topics
Authors
Recent
Search
2000 character limit reached

Yat Kernel: Geometric & Neural Computation

Updated 5 July 2026
  • Yat Kernel is a rational similarity operator that fuses quadratic alignment with distance-based regularization to capture localized geometric information in Euclidean spaces.
  • It serves as a Mercer kernel with a unique RKHS and universal approximation property when the bias parameter is positive, linking polynomial alignment with inverse-multiquadric kernels.
  • Employed in Neural Matter Networks and Yat-attention, the kernel replaces conventional activation functions with intrinsic self-normalization for stable and efficient training.

Searching arXiv for the cited Yat kernel papers to ground the article in the latest preprints. to=functions.arxiv_search 彩神争霸的 json {"query":"\"Yat kernel\" OR \"yat-product\" OR \"Neural Matter Networks\" OR \"A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance\" OR \"No More DeLuLu: Physics-Inspired Kernel Networks for Geometrically-Grounded Neural Computation\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} to=functions.arxiv_search 天天中彩票公众号json {"query":"ti:\"No More DeLuLu\" OR ti:\"Artificial Neurons That Matter\" OR ti:\"A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distance\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} to=functions.arxiv_search ացինjson {"query":"all:\"(Bouhsine, 22 Feb 2026)\" OR all:\"(Bouhsine, 2024)\" OR all:\"(Bouhsine, 5 May 2026)\"","max_results":10,"sort_by":"submittedDate","sort_order":"descending"} The Yat Kernel is a rational similarity operator, also called the Yat-product or E-product, defined on vectors in a common Euclidean space by a squared alignment term in the numerator and a squared-distance term in the denominator. In its stabilized form, it is written

$k_{\E}(\mathbf{x},\mathbf{w})=\frac{(\mathbf{x}^\top \mathbf{w})^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad \varepsilon>0,$

and in bias-augmented form

kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.

Across recent work, it has been proposed both as a geometric primitive for activation-free neural layers and as a Mercer kernel with a unique reproducing kernel Hilbert space (RKHS), universality on compact domains when b>0b>0, and an explicit connection to inverse-multiquadric (IMQ) kernels (Bouhsine, 2024, Bouhsine, 22 Feb 2026, Bouhsine, 5 May 2026).

1. Definition and geometric content

The core Yat-product combines quadratic alignment with inverse-square proximity: $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$ The numerator

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^2

is large when w\mathbf{w} and x\mathbf{x} are strongly aligned and vanishes when they are orthogonal. Because of the square, the score is sign-invariant: aligned and anti-aligned vectors with the same magnitude of inner product yield the same value. The denominator

r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon

imposes locality, so the response decays as distance grows. Using

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},

the same operator can be written as

$\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$

which makes explicit that the denominator depends on both norms and alignment (Bouhsine, 22 Feb 2026).

A bias-augmented form is used inside Neural Matter Networks: kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.0 A later RKHS treatment standardizes this as

kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.1

with both arguments in the same ambient space, kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.2 acting as a trainable center or weight and kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.3 as the input (Bouhsine, 5 May 2026).

The physics analogy used in the literature is that a Yat unit behaves like a potential well centered at kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.4: strong response requires both alignment and proximity. This differs from a linear neuron, whose response is globally controlled by a hyperplane, and from a purely radial kernel, whose response depends only on distance (Bouhsine, 22 Feb 2026).

2. From pseudo-metric formulation to Mercer kernel

The earliest neural-network formulation frames the operator as one of Bouhsine’s two products. For vectors kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.5,

kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.6

With

kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.7

that work states that kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.8 is semi-metric, while kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.9 is pseudo-metric. It also emphasizes that the operator is symmetric and non-negative but not associative or distributive, and that it is “not presented as a PSD kernel” or as an inner product of a feature map (Bouhsine, 2024).

Subsequent work regularizes the denominator by introducing b>0b>00 and shifts the mathematical status of the object. A key identity is

b>0b>01

Hence

b>0b>02

This realizes the Yat kernel as a nonnegative mixture of products of a quadratic polynomial kernel and a Gaussian radial kernel. On every compact set b>0b>03, the kernel is then proved symmetric, continuous, and positive definite, and therefore a Mercer kernel on b>0b>04 (Bouhsine, 22 Feb 2026).

The 2026 RKHS treatment sharpens the parameter dependence. For

b>0b>05

the kernel is PSD for every b>0b>06, but the sign condition is stated to be sharp: for negative b>0b>07, the numerator b>0b>08 can be indefinite (Bouhsine, 5 May 2026).

This development resolves an important historical tension. The 2024 neural-network presentation emphasizes pseudo-metric structure and the lack of conventional algebraic properties, whereas the 2026 analyses establish Mercer, RKHS, and universality results for the stabilized kernel under explicit parameter conditions (Bouhsine, 2024, Bouhsine, 22 Feb 2026, Bouhsine, 5 May 2026).

3. RKHS structure, universality, and relation to IMQ kernels

For every compact b>0b>09, positive definiteness yields a unique RKHS $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$0 and feature map $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$1 such that

$\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$2

The corresponding existence-and-uniqueness statement is given explicitly via the Moore–Aronszajn theorem, with uniqueness understood up to isometry (Bouhsine, 22 Feb 2026).

The later RKHS analysis isolates a decisive distinction between $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$3 and $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$4. The kernel factors as

$\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$5

where

$\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$6

Thus Yat is the product of a degree-2 polynomial alignment kernel and an IMQ radial kernel. Expanding the numerator gives three channels,

$\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$7

with

$\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$8

Accordingly, the Yat RKHS decomposes into a radial IMQ channel, a linear-alignment IMQ channel, and a quadratic-alignment IMQ channel. At $\E(\mathbf{w},\mathbf{x}) := \frac{\langle \mathbf{w}, \mathbf{x} \rangle^2}{\|\mathbf{w} - \mathbf{x}\|^2 + \varepsilon}.$9, the radial and linear channels vanish; only the quadratic-alignment channel remains (Bouhsine, 5 May 2026).

For A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^20, the kernel Loewner-dominates a scaled IMQ: A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^21 By Aronszajn’s inclusion theorem,

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^22

Because IMQ is universal on compact subsets of A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^23, the Yat RKHS is also universal when A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^24. On compact domains, this yields characteristicness and strict positive definiteness as well (Bouhsine, 5 May 2026).

A central algebraic result connects Yat atoms to IMQ atoms exactly. With

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^25

one has

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^26

Thus a second finite difference in the bias recovers any IMQ atom from three positive-bias Yat atoms exactly, and the paper states that this is sharp at three atoms in every dimension for exact pointwise equality at generic nonzero centers (Bouhsine, 5 May 2026).

The same work also isolates a nonradial asymptotic channel absent from finite IMQ expansions. For the directional far-field trace,

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^27

one obtains

A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^28

whereas finite IMQ combinations decay to zero. This separates Yat’s polynomial alignment channels from purely radial IMQ behavior (Bouhsine, 5 May 2026).

4. Neural Matter Networks and Yat-based attention

The Yat Kernel is the defining primitive of the Neural Matter Network (NMN), in which it replaces the conventional sequence “linear projection + activation + normalization.” For an input A(w,x)=(w⊤x)2A(\mathbf{w},\mathbf{x})=(\mathbf{w}^\top \mathbf{x})^29, an NMN layer with w\mathbf{w}0 units is written

w\mathbf{w}1

where the scaling factor is learned via

w\mathbf{w}2

with w\mathbf{w}3 learnable. In this design, the Yat-product is the sole non-linearity (Bouhsine, 22 Feb 2026).

An earlier implementation paper gives the layer computation explicitly in Flax/JAX. For inputs w\mathbf{w}4 and weight matrix w\mathbf{w}5, it first computes dot products w\mathbf{w}6, then distances

w\mathbf{w}7

and finally

w\mathbf{w}8

No ReLU, GeLU, or tanh is applied in hidden layers; “all hidden layers are linear-to-yat only” (Bouhsine, 2024).

The 2026 framework extends the same primitive to transformers. In Aether-GPT2, scaled dot-product attention is replaced by Yat-attention: w\mathbf{w}9 followed by

x\mathbf{x}0

The MLP block is replaced by NMN x\mathbf{x}1 + Linear x\mathbf{x}2, and the paper states that there is no LayerNorm in blocks because the denominator provides intrinsic norm control; attempts to reintroduce LayerNorm caused divergence (Bouhsine, 22 Feb 2026).

The same paper proves a universal approximation theorem: for compact x\mathbf{x}3, the span of atoms

x\mathbf{x}4

is dense in x\mathbf{x}5 under the uniform norm. The proof route uses finite-difference identities in x\mathbf{x}6 to express IMQ atoms as linear combinations of Yat atoms, together with the universality of IMQ (Bouhsine, 22 Feb 2026).

5. Regularization, gradients, and representational geometry

A central claim of the Yat literature is that normalization is moved into the kernel. Because

x\mathbf{x}7

the denominator grows like x\mathbf{x}8 for large x\mathbf{x}9, counteracting numerator growth and yielding bounded, scale-aware responses. The 2026 NMN paper describes this as self-normalization and self-regulation, in contrast with architectures that rely on BatchNorm or LayerNorm (Bouhsine, 22 Feb 2026).

The same work proves analyticity and a bounded-domain Lipschitz estimate. For fixed r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon0 and r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon1, the map r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon2 is Lipschitz on the unit ball with constant

r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon3

so if r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon4,

r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon5

It also gives the asymptotic bound

r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon6

showing that the response converges to a finite constant along rays (Bouhsine, 22 Feb 2026).

The gradient formula is explicit: r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon7 Far from the center,

r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon8

so outliers produce negligible gradients. The paper explicitly describes this as outlier down-weighting and reports empirically stable training in BF16 without gradient clipping (Bouhsine, 22 Feb 2026).

An information-geometric interpretation is also developed. Because the kernel induces an RKHS, each representation carries a Hilbertian inner product and norm. The literature emphasizes three geometric features: orthogonality sensitivity, localized potential wells, and an NTK whose entries decay strongly for orthogonal inputs. One paper states that the r(w,x)=∥w−x∥2+εr(\mathbf{w},\mathbf{x})=\|\mathbf{w}-\mathbf{x}\|^2+\varepsilon9-NTK eigenvalues decay as ∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},0, compared with ∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},1 for the ReLU NTK, and interprets this as yielding smoother function spaces and better-conditioned learning (Bouhsine, 22 Feb 2026).

For shared-∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},2 Yat layers, the RKHS norm is available in closed form. If

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},3

then

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},4

The diagonal term is

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},5

and the corresponding empirical Rademacher complexity bound stated for the RKHS ball ∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},6 is

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},7

when ∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},8 (Bouhsine, 5 May 2026).

6. Empirical results, comparisons, and open questions

On MNIST, the 2026 NMN paper studies a 10-neuron prototype classifier with logits

∥w−x∥2=∥w∥2+∥x∥2−2w⊤x,\|\mathbf{w}-\mathbf{x}\|^2=\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x},9

Training uses Adam, learning rate $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$0, and 5 epochs. The reported comparison is as follows (Bouhsine, 22 Feb 2026).

Model Test Acc. $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$1
Linear 92.08% +13.8%
Yat 92.38% −4.5%

The same experiment is used to support the claim of bounded prototype evolution: linear prototype norms grow, whereas Yat prototype norms shrink slightly. The paper also reports a superposition-style sign-flip test, exploiting the sign invariance of the squared numerator:

Model Original Acc. Acc. after $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$2
Linear 92.04% 0.01%
Yat 92.18% 87.87%

The language-modeling results are reported for GPT-2 “small” and Aether-GPT2, both at roughly $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$3M parameters, trained on 2.5B tokens from FineWeb in BF16 with AdamW and the same batch and context length (Bouhsine, 22 Feb 2026).

Quantity GPT-2 Aether-GPT2
Final train loss 4.1969 4.0479
Final val loss 4.6417 4.5747
Memory – −15–25%

The paper states a +1.45% validation improvement and attributes the memory reduction to dropping LayerNorm activations. It also reports better calibration curves for Aether-GPT2 and stable BF16 training without gradient clipping or LayerNorm (Bouhsine, 22 Feb 2026).

The earlier 2024 NMN paper reports broader vision benchmarks with Yat-based MLP and ViT variants. On CIFAR-10, CIFAR-100, Caltech101, Oxford Flowers, and STL-10, the summary states that E-neuron models are “consistently better or comparable” to traditional MLP or ViT baselines, while emphasizing that the experiments remain on mid-scale vision tasks and that behavior on massive-scale NLP or structured sequential data is open (Bouhsine, 2024).

The 2026 RKHS paper adds proof-of-concept experiments outside the original NMN formulation: a single Yat layer on frozen CLIP ViT-B/32 features for ImageNet-1k, and a 262M-parameter causal LM in which MLP blocks are replaced by Yat blocks. That paper describes these as proof-of-concept results rather than heavily tuned benchmarks and uses them to support the claim that the single-layer RKHS theory is not vacuous (Bouhsine, 5 May 2026).

Several open questions are stated explicitly across the literature. One concerns depth: there is “no global, parameter-independent kernel that exactly captures a deep Yat stack’s behaviour,” and a conjectural treatment via pullback kernels is suggested (Bouhsine, 5 May 2026). Another concerns the choice of $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$4; one paper notes an optimal scaling $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$5 for noisy inputs, while also presenting practical sensitivity to $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$6 and $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$7 (Bouhsine, 22 Feb 2026, Bouhsine, 5 May 2026). Computationally, early analyses estimate an efficiency ratio

$\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$8

relative to dot product plus ReLU, whereas a later optimized analysis gives per-neuron counts of $\E(\mathbf{w}, \mathbf{x})= \frac{(\mathbf{w}^\top\mathbf{x})^2}{\|\mathbf{w}\|^2+\|\mathbf{x}\|^2-2\mathbf{w}^\top\mathbf{x}+\varepsilon},$9 for Yat versus kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.00 for Linear+ReLU, with the same asymptotic kb,ε(w,x)=(w⊤x+b)2∥x−w∥2+ε,b≥0, ε>0.k_{b,\varepsilon}(\mathbf{w},\mathbf{x})=\frac{(\mathbf{w}^\top\mathbf{x}+b)^2}{\|\mathbf{x}-\mathbf{w}\|^2+\varepsilon}, \qquad b\ge 0,\ \varepsilon>0.01 forward and backward complexity (Bouhsine, 2024, Bouhsine, 22 Feb 2026).

Taken together, these works define the Yat Kernel as a geometrically grounded hidden-unit primitive whose later formulations place it within standard kernel theory without abandoning its original emphasis on alignment, orthogonality, locality, and prototype-like neural computation (Bouhsine, 2024, Bouhsine, 22 Feb 2026, Bouhsine, 5 May 2026).

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Yat Kernel.