---
title: Projective Kolmogorov-Arnold Networks (P-KANs)
url: https://www.emergentmind.com/topics/projective-kolmogorov-arnold-networks-p-kans
type: topic
---

# Projective Kolmogorov-Arnold Networks (P-KANs)

Searching arXiv for the target paper and closely related KAN work.
First, retrieving the target paper by arXiv ID.
Now searching for the original KAN paper for minimal contextual grounding.
Projective Kolmogorov-Arnold Networks (P-KANs) are a training framework for Kolmogorov-Arnold Networks (KANs) that preserves spline-based edge parameterizations while steering each learned edge function toward a lower-dimensional, interpretable functional space selected by an entropy criterion. In the formulation presented in "Projective Kolmogorov Arnold Neural Networks (P-KANs): Entropy-Driven Functional Space Discovery for Interpretable Machine Learning" [2509.20049], the method is motivated by redundancy in high-dimensional spline parameter spaces, where many distinct spline coefficient configurations can realize nearly the same function. P-KANs therefore treat spline space as a flexible search space and use entropy-minimizing projections into candidate spaces such as Fourier, Chebyshev, and Bessel to compress edge functions, reduce nuisance directions in the Jacobian, improve robustness, and enable mixed functional representations across edges.

## 1. Redundancy in spline-based KANs

Standard KAN layers are described by
$$
y_j = \sum_{i=1}^{n} f_{ij}(x_i) + b_j,
$$
with nonlinear edge functions \(f_{ij}\) typically parameterized as splines [2509.20049]. The central motivation for P-KANs is that these spline spaces are large and overcomplete for many tasks. A degree-\(k\) spline with \(p\) control points can realize many functionally equivalent shapes, so the parameterization contains substantial slack.

The paper characterizes this slack through the Jacobian of the network with respect to parameters,
$$
J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.
$$
Many small Jacobian entries indicate directions in parameter space that barely affect the output. Following the nuisance-space interpretation adopted in the paper, such low-sensitivity directions are where overfitting and poor generalization can hide: the model may absorb noise or training-set idiosyncrasies without converging to a compressive or meaningful representation [2509.20049].

This diagnosis places P-KANs within a specific critique of spline-based KAN practice. The issue is not that splines are insufficiently expressive; rather, they can be excessively expressive relative to the latent regularity of the target function. The proposed remedy is therefore not to remove spline flexibility, but to use it as a search mechanism and then discover whether a simpler functional basis yields a more concentrated representation.

## 2. Entropy-driven discovery of functional spaces

The core mechanism is adapted from signal analysis and best-basis or dictionary-learning intuition. If a function is naturally represented in a given basis, its coefficient vector in that basis should be concentrated rather than diffuse. For an edge spline \(s_{l,i,j}(x)\), projection into a candidate orthogonal functional space \(\mathcal{R}\) is written as
$$
s(x) \rightarrow s^{r}(x) = \sum_q \alpha_q r^q(x),
$$
where \(r^q\) denotes basis functions and \(\alpha_q\) the corresponding coefficients [2509.20049].

The coefficients are normalized by absolute amplitude,
$$
\hat{\alpha}_q = \frac{|\alpha_q|}{\sum_t |\alpha_t|},
$$
and the representation entropy is defined as
$$
E_{\mathcal{R}} = -\sum_q \hat{\alpha}_q \log(\hat{\alpha}_q).
$$
Low entropy indicates that a small number of coefficients dominate, so the edge function is simple in that basis. The paper explicitly contrasts this absolute-amplitude entropy with coefficient-energy entropy based on squared magnitudes, arguing that the former promotes sparsity rather than spreading energy across many basis functions [2509.20049].

The method is described metaphorically through a "gravitational term": entropy minimization pulls the spline away from generic high-dimensional spline space and toward a lower-dimensional "shadow" space where the representation is compact and interpretable. If an edge is nearly sinusoidal, Fourier coefficients become sparse; if it exhibits smooth polynomial structure, Chebyshev coefficients become compact; if radial or oscillatory behavior is more suitable, Bessel-like bases may dominate. This implies that P-KANs do not impose a fixed functional family a priori. Instead, they use entropy as a criterion for functional-space discovery.

## 3. Objective, selection rule, and adaptive replacement

The training objective combines task loss, entropy minimization, and edge regularization. In the short implementation summary, the paper gives
$$
Q = \alpha \cdot \text{loss} + \beta \cdot \overline{E^*} + \gamma \cdot \text{regularization},
$$
where \(\alpha\) weights the standard reconstruction or task loss, \(\beta\) weights the entropy-driven basis-selection term, \(\gamma\) weights edge regularization, and the overline denotes averaging over all edges [2509.20049]. The fuller description also specifies \(\Omega_{l,i,j}\) as sampled points along edge \((l,i,j)\) and \(\ell^m\) as the chosen \(m\)-norm in the edge regularizer.

Multiple candidate spaces are considered simultaneously. If there are \(m\) candidate spaces, their entropies are gathered as
$$
E_{\Sigma} = [E_{\mathcal{R}_0}, E_{\mathcal{R}_1}, \ldots, E_{\mathcal{R}_m}].
$$
The optimal-space score \(E^*\) is then obtained through a differentiable softmin-like approximation to \(\min_k E_{\mathcal{R}_k}\), with a temperature-like parameter \(\lambda\) that is increased gradually so that early training is not biased by random initialization [2509.20049].

Projection quality is assessed with \(R^2\),
$$
R^{2} = 1 - \frac{\sum_{i}\left( y_{i} - F(x_{i})\right)^2}{\sum_{i}\left( y_{i} - \bar{y}\right)^2}.
$$
The reported workflow is sequential. The network is first trained in spline space under the unified objective; each edge is then projected into the candidate spaces; coefficients and entropies are computed; the minimum-entropy basis is selected subject to an \(R^2\) threshold; edges whose best fit exceeds \(R^2_{\min}\) are replaced by fixed parametric functions and frozen; the network is fine-tuned; and a regret criterion can revert fixed edges back to splines if retraining degrades performance too much [2509.20049].

A common misunderstanding is that P-KANs commit to parametric bases from the outset. The reported procedure does not do this. The spline representation remains the initial and flexible substrate, while projection, \(R^2\), and regret act as discovery and safety mechanisms.

## 4. Candidate bases and mixed functional representations

The implementation described in the paper uses three candidate families: Fourier space, Chebyshev polynomials, and Bessel functions [2509.20049]. These are treated discretely: splines are evaluated on grid points, and projections are computed using transform-like procedures, including DFT for Fourier and torch-dct-style tools for Chebyshev coefficient identification. Bessel representations are described as using gamma-function expansions with trigonometric approximations.

A central empirical result is that there is no single universally best basis. Different edges can converge to different optimal spaces depending on the local relationship they encode. The paper terms this phenomenon "mixed functional representations." An edge capturing oscillatory structure may become Fourier-like; an edge encoding smooth polynomial behavior may become Chebyshev-like; another may remain spline-like if none of the candidate projective spaces provides an adequate fit [2509.20049].

This per-edge specialization is important because the method is not framed as global basis replacement. It is instead a heterogeneous representation strategy in which different subcomponents of the same network may settle into different mathematical regularities. The paper argues that this improves interpretability and often yields better compression than forcing all edges into the same basis.

## 5. Empirical behavior across synthetic, PDE, and industrial tasks

The ablation study covers several synthetic function types: simple sinusoidal, Gaussian, discontinuous, oscillatory \((\sin\) and \(\cos\) combination), and polynomial (cubic). Data are sampled at random coordinates in compact domains rather than on regular grids, and baselines include standard spline KANs. Reported metrics include \(R^2\), log-loss, parameter count, Jacobian spectral spread or variance, and training behavior. The principal findings are that P-KAN maintains high \(R^2\) across most model complexities, standard KANs show more instability and outliers, and parameter count per edge is reduced from cubic spline with 20 control points, that is \(n=23\) degrees of freedom per edge, to \(n=4\) parameters in the projective representation. The paper describes this as over 80% reduction and later summarizes it as 83% reduction [2509.20049].

Robustness experiments use 1D and 2D PDE benchmarks: the 1D wave equation and the 2D heat equation. Noise types are Gaussian, uniform, and salt-and-pepper; SNR levels are 5, 10, 15, 20, and 30 dB, plus a clean baseline; architectures range from minimal \([1,1]\) through tiny \([1,2,1]\), small \([2,4,1]\), and medium \([2,8,1]\); and each experiment is repeated 5 times. The key reported result is that P-KAN generalizes much better under noise and random sampling. The paper states that KAN validation loss can exceed \(10^{13}\) in these settings, while P-KAN stays around \(<\mathcal{O}(10^{-1})\); the summary also mentions \(<\mathcal{-1}(10^{-3})\), which the paper itself presents in a typographically inconsistent form but clearly intends as an extremely small loss scale. The same section reports that P-KAN trains more slowly, with projection and basis-selection steps incurring about a \(12.4\times\) training-time cost [2509.20049].

The industrial case study concerns automated fibre placement with laser-line scanner data. The task is to predict the next laser scan from a \(5\times 5\) input patch. The training set contains only 14 samples from 5 tow placements, spanning single-tow, parallel-tow, and overlapped-tow cases. The network is described as a \(25 \rightarrow 5\) model taking flattened \(5 \times 5\) inputs and predicting adjacent next-scan values. Despite severe noise and limited data, the reported P-KAN model predicts tow placement accurately, including an unseen wrinkled sample, the octogonal test piece, and multi-layer configurations. The reported mode RMSE is around \(\sim 0.08\) on the octogonal test piece, and the dominant learned representation is reported to be Fourier-like [2509.20049].

## 6. Diagnostics, practical scope, and limitations

The paper reports several diagnostics: model complexity versus \(R^2\), functional-space ablation, log-loss curves, spectral or Jacobian analysis, hyperparameter sweeps over \(\alpha,\beta,\gamma\), noise robustness plots, and training-time comparison [2509.20049]. A notable diagnostic claim is that after projection, Jacobian variance or spread increases substantially, which is interpreted as reduced redundancy and less nuisance-space occupancy. Hyperparameter studies also identify a threshold effect around \(\beta < 0.5\gamma\), where entropy minimization is too weak relative to regularization to force many edges into projective form.

The reported practical upside is strongest when data are noisy, data are scarce, interpretability matters, and the underlying relationships may be expressible in known functional families. The AFP example is presented as the clearest instance of this regime. The paper’s bottom-line characterization is that P-KANs function as a compression-and-discovery layer for KANs: they trade additional computation for better robustness, better interpretability, and smaller effective parameterization [2509.20049].

The main limitations are also explicit. Projection across multiple spaces is computationally expensive; the implementation is restricted to classical spaces such as Fourier, Chebyshev, and Bessel; the balance among \(\alpha,\beta,\gamma\) is nontrivial and may require empirical tuning; orthonormal projection can separate shape from magnitude in ways that matter for amplitude-sensitive applications; and projection cost grows roughly linearly with the number of edges, making very large models less attractive than standard KANs when speed is the primary objective [2509.20049].

Taken together, these constraints delimit the intended use of P-KANs. They are not presented as a universal replacement for standard KAN training. Rather, they are designed for settings in which discoverable low-dimensional functional structure is present and exploiting that structure can justify the additional optimization and projection overhead.

Source: https://www.emergentmind.com/topics/projective-kolmogorov-arnold-networks-p-kans