Projective Kolmogorov-Arnold Networks (P-KANs)

• Projective Kolmogorov-Arnold Networks (P-KANs) are a training framework that compresses high-dimensional, overcomplete spline representations into interpretable, low-dimensional functional spaces.
• The method projects learned spline edge functions into candidate bases such as Fourier, Chebyshev, and Bessel, using entropy minimization to identify the most compact representation.
• P-KANs balance task loss, entropy-driven basis selection, and regularization to improve robustness, reduce parameter redundancy, and achieve significant compression with maintained R² performance.

Searching arXiv for the target paper and closely related KAN work. First, retrieving the target paper by arXiv ID. Now searching for the original KAN paper for minimal contextual grounding. Projective Kolmogorov-Arnold Networks (P-KANs) are a training framework for Kolmogorov-Arnold Networks (KANs) that preserves spline-based edge parameterizations while steering each learned edge function toward a lower-dimensional, interpretable functional space selected by an entropy criterion. In the formulation presented in "Projective Kolmogorov Arnold Neural Networks (P-KANs): Entropy-Driven Functional Space Discovery for Interpretable Machine Learning" (Poole et al., 24 Sep 2025), the method is motivated by redundancy in high-dimensional spline parameter spaces, where many distinct spline coefficient configurations can realize nearly the same function. P-KANs therefore treat spline space as a flexible search space and use entropy-minimizing projections into candidate spaces such as Fourier, Chebyshev, and Bessel to compress edge functions, reduce nuisance directions in the Jacobian, improve robustness, and enable mixed functional representations across edges.

1. Redundancy in spline-based KANs

Standard KAN layers are described by

$y_j = \sum_{i=1}^{n} f_{ij}(x_i) + b_j,$

with nonlinear edge functions $f_{ij}$ typically parameterized as splines (Poole et al., 24 Sep 2025). The central motivation for P-KANs is that these spline spaces are large and overcomplete for many tasks. A degree-$k$ spline with $p$ control points can realize many functionally equivalent shapes, so the parameterization contains substantial slack.

The paper characterizes this slack through the Jacobian of the network with respect to parameters,

$J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$

Many small Jacobian entries indicate directions in parameter space that barely affect the output. Following the nuisance-space interpretation adopted in the paper, such low-sensitivity directions are where overfitting and poor generalization can hide: the model may absorb noise or training-set idiosyncrasies without converging to a compressive or meaningful representation (Poole et al., 24 Sep 2025).

This diagnosis places P-KANs within a specific critique of spline-based KAN practice. The issue is not that splines are insufficiently expressive; rather, they can be excessively expressive relative to the latent regularity of the target function. The proposed remedy is therefore not to remove spline flexibility, but to use it as a search mechanism and then discover whether a simpler functional basis yields a more concentrated representation.

2. Entropy-driven discovery of functional spaces

The core mechanism is adapted from signal analysis and best-basis or dictionary-learning intuition. If a function is naturally represented in a given basis, its coefficient vector in that basis should be concentrated rather than diffuse. For an edge spline $s_{l,i,j}(x)$, projection into a candidate orthogonal functional space $\mathcal{R}$ is written as

$$s(x) \rightarrow s^{r}(x) = \sum_q \alpha_q r^q(x),$$

where $r^q$ denotes basis functions and $\alpha_q$ the corresponding coefficients (Poole et al., 24 Sep 2025).

The coefficients are normalized by absolute amplitude,

$f_{ij}$0

and the representation entropy is defined as

$f_{ij}$1

Low entropy indicates that a small number of coefficients dominate, so the edge function is simple in that basis. The paper explicitly contrasts this absolute-amplitude entropy with coefficient-energy entropy based on squared magnitudes, arguing that the former promotes sparsity rather than spreading energy across many basis functions (Poole et al., 24 Sep 2025).

The method is described metaphorically through a "gravitational term": entropy minimization pulls the spline away from generic high-dimensional spline space and toward a lower-dimensional "shadow" space where the representation is compact and interpretable. If an edge is nearly sinusoidal, Fourier coefficients become sparse; if it exhibits smooth polynomial structure, Chebyshev coefficients become compact; if radial or oscillatory behavior is more suitable, Bessel-like bases may dominate. This implies that P-KANs do not impose a fixed functional family a priori. Instead, they use entropy as a criterion for functional-space discovery.

3. Objective, selection rule, and adaptive replacement

The training objective combines task loss, entropy minimization, and edge regularization. In the short implementation summary, the paper gives

$f_{ij}$2

where $f_{ij}$3 weights the standard reconstruction or task loss, $f_{ij}$4 weights the entropy-driven basis-selection term, $f_{ij}$5 weights edge regularization, and the overline denotes averaging over all edges (Poole et al., 24 Sep 2025). The fuller description also specifies $f_{ij}$6 as sampled points along edge $f_{ij}$7 and $f_{ij}$8 as the chosen $f_{ij}$9-norm in the edge regularizer.

Multiple candidate spaces are considered simultaneously. If there are $k$0 candidate spaces, their entropies are gathered as

$k$1

The optimal-space score $k$2 is then obtained through a differentiable softmin-like approximation to $k$3, with a temperature-like parameter $k$4 that is increased gradually so that early training is not biased by random initialization (Poole et al., 24 Sep 2025).

Projection quality is assessed with $k$5,

$k$6

The reported workflow is sequential. The network is first trained in spline space under the unified objective; each edge is then projected into the candidate spaces; coefficients and entropies are computed; the minimum-entropy basis is selected subject to an $k$7 threshold; edges whose best fit exceeds $k$8 are replaced by fixed parametric functions and frozen; the network is fine-tuned; and a regret criterion can revert fixed edges back to splines if retraining degrades performance too much (Poole et al., 24 Sep 2025).

A common misunderstanding is that P-KANs commit to parametric bases from the outset. The reported procedure does not do this. The spline representation remains the initial and flexible substrate, while projection, $k$9, and regret act as discovery and safety mechanisms.

4. Candidate bases and mixed functional representations

The implementation described in the paper uses three candidate families: Fourier space, Chebyshev polynomials, and Bessel functions (Poole et al., 24 Sep 2025). These are treated discretely: splines are evaluated on grid points, and projections are computed using transform-like procedures, including DFT for Fourier and torch-dct-style tools for Chebyshev coefficient identification. Bessel representations are described as using gamma-function expansions with trigonometric approximations.

A central empirical result is that there is no single universally best basis. Different edges can converge to different optimal spaces depending on the local relationship they encode. The paper terms this phenomenon "mixed functional representations." An edge capturing oscillatory structure may become Fourier-like; an edge encoding smooth polynomial behavior may become Chebyshev-like; another may remain spline-like if none of the candidate projective spaces provides an adequate fit (Poole et al., 24 Sep 2025).

This per-edge specialization is important because the method is not framed as global basis replacement. It is instead a heterogeneous representation strategy in which different subcomponents of the same network may settle into different mathematical regularities. The paper argues that this improves interpretability and often yields better compression than forcing all edges into the same basis.

5. Empirical behavior across synthetic, PDE, and industrial tasks

The ablation study covers several synthetic function types: simple sinusoidal, Gaussian, discontinuous, oscillatory $p$0 and $p$1 combination), and polynomial (cubic). Data are sampled at random coordinates in compact domains rather than on regular grids, and baselines include standard spline KANs. Reported metrics include $p$2, log-loss, parameter count, Jacobian spectral spread or variance, and training behavior. The principal findings are that P-KAN maintains high $p$3 across most model complexities, standard KANs show more instability and outliers, and parameter count per edge is reduced from cubic spline with 20 control points, that is $p$4 degrees of freedom per edge, to $p$5 parameters in the projective representation. The paper describes this as over 80% reduction and later summarizes it as 83% reduction (Poole et al., 24 Sep 2025).

Robustness experiments use 1D and 2D PDE benchmarks: the 1D wave equation and the 2D heat equation. Noise types are Gaussian, uniform, and salt-and-pepper; SNR levels are 5, 10, 15, 20, and 30 dB, plus a clean baseline; architectures range from minimal $p$6 through tiny $p$7, small $p$8, and medium $p$9; and each experiment is repeated 5 times. The key reported result is that P-KAN generalizes much better under noise and random sampling. The paper states that KAN validation loss can exceed $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$0 in these settings, while P-KAN stays around $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$1; the summary also mentions $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$2, which the paper itself presents in a typographically inconsistent form but clearly intends as an extremely small loss scale. The same section reports that P-KAN trains more slowly, with projection and basis-selection steps incurring about a $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$3 training-time cost (Poole et al., 24 Sep 2025).

The industrial case study concerns automated fibre placement with laser-line scanner data. The task is to predict the next laser scan from a $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$4 input patch. The training set contains only 14 samples from 5 tow placements, spanning single-tow, parallel-tow, and overlapped-tow cases. The network is described as a $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$5 model taking flattened $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$6 inputs and predicting adjacent next-scan values. Despite severe noise and limited data, the reported P-KAN model predicts tow placement accurately, including an unseen wrinkled sample, the octogonal test piece, and multi-layer configurations. The reported mode RMSE is around $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$7 on the octogonal test piece, and the dominant learned representation is reported to be Fourier-like (Poole et al., 24 Sep 2025).

6. Diagnostics, practical scope, and limitations

The paper reports several diagnostics: model complexity versus $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$8, functional-space ablation, log-loss curves, spectral or Jacobian analysis, hyperparameter sweeps over $J_{i,j} = \frac{\partial f_i}{\partial \theta_j}.$9, noise robustness plots, and training-time comparison (Poole et al., 24 Sep 2025). A notable diagnostic claim is that after projection, Jacobian variance or spread increases substantially, which is interpreted as reduced redundancy and less nuisance-space occupancy. Hyperparameter studies also identify a threshold effect around $s_{l,i,j}(x)$0, where entropy minimization is too weak relative to regularization to force many edges into projective form.

The reported practical upside is strongest when data are noisy, data are scarce, interpretability matters, and the underlying relationships may be expressible in known functional families. The AFP example is presented as the clearest instance of this regime. The paper’s bottom-line characterization is that P-KANs function as a compression-and-discovery layer for KANs: they trade additional computation for better robustness, better interpretability, and smaller effective parameterization (Poole et al., 24 Sep 2025).

The main limitations are also explicit. Projection across multiple spaces is computationally expensive; the implementation is restricted to classical spaces such as Fourier, Chebyshev, and Bessel; the balance among $s_{l,i,j}(x)$1 is nontrivial and may require empirical tuning; orthonormal projection can separate shape from magnitude in ways that matter for amplitude-sensitive applications; and projection cost grows roughly linearly with the number of edges, making very large models less attractive than standard KANs when speed is the primary objective (Poole et al., 24 Sep 2025).

Taken together, these constraints delimit the intended use of P-KANs. They are not presented as a universal replacement for standard KAN training. Rather, they are designed for settings in which discoverable low-dimensional functional structure is present and exploiting that structure can justify the additional optimization and projection overhead.