Kolmogorov-Arnold Networks Overview

• Kolmogorov-Arnold Networks are neural architectures rooted in the theorem that exactly represents multivariate functions as sums of univariate functions.
• They employ edge-based learnable functions using spline, Fourier, or sinusoidal bases to achieve superior parameter efficiency and interpretability compared to traditional MLPs.
• Empirical and theoretical studies indicate that KANs can achieve dimension-independent error bounds and sample complexity, making them ideal for high-dimensional scientific applications.

Kolmogorov-Arnold Networks (KANs) are a class of neural architectures rooted in the Kolmogorov-Arnold superposition theorem, which provides a constructive, exact representation of multivariate continuous functions via compositions and sums of univariate functions. KANs have rapidly gained prominence as theoretically grounded and practically effective alternatives to multilayer perceptrons (MLPs), offering advantages in parameter efficiency, scaling, interpretability, and function approximation flexibility. The following sections synthesize the mathematical foundation, architectural design, approximation theory, computational aspects, empirical performance, extensions, and future directions of KANs as delineated in recent literature.

1. Mathematical Foundation: The Kolmogorov-Arnold Representation

The core mathematical underpinning of KANs is the Kolmogorov-Arnold representation theorem, which states that any continuous function $f : [0,1]^d \to \mathbb{R}$ admits an exact decomposition of the form

$$f(x_1, \ldots, x_d) = \sum_{q=1}^{2d+1} \Phi_q\left(\sum_{p=1}^d \phi_{q,p}(x_p)\right)$$

where $\phi_{q,p}$ and $\Phi_q$ are continuous univariate functions. This contrasts with the universal approximation theorem for MLPs, which provides only an approximate representation and no fixed-size construction. Modern KANs operationalize this theorem by parameterizing each univariate function as a flexible, learnable mapping—typically using spline bases such as B-splines or efficient alternatives such as sinusoidal or Fourier bases (Liu et al., 30 Apr 2024, Gleyzer et al., 1 Aug 2025).

2. Structural and Algorithmic Innovations

KANs differ fundamentally from classical MLPs in several respects:

• Edge-Based Learnable Functions: In standard MLPs, the nonlinearity is fixed (e.g., ReLU) at the nodes, with scalar weights on edges. In KANs, each edge is associated with a learnable 1D function $\phi_{l,j,i} : \mathbb{R} \to \mathbb{R}$, applied directly to the source node output before summing at the destination node:

$x_{l+1,j} = \sum_{i=1}^{n_l} \phi_{l,j,i}(x_{l,i})$

• Spline and Non-Spline Parameterizations: Early KANs use B-splines for their localization and theoretical properties; later variants also employ Chebyshev polynomials, radial basis functions, weighted sinusoids, and even parameter-adaptive mixtures for their univariate functions (Li, 10 May 2024, Gleyzer et al., 1 Aug 2025).
• Grid Extension and Adaptivity: The nonlinearity on edges is typically controlled via grids over the input domain that may be dynamically refined or adapted during training to better capture local or non-smooth structure (Liu et al., 30 Apr 2024).
• Residual and Skip Connections: Recent architectures utilize skip connections and gating:

${\mathcal L}(x) = \sigma_{\beta:I}(Ax+b) + Gx$

where $G$ is a gating matrix and $\sigma_{\beta:I}$ is a spline-based activation, mirroring modern deep residual architectures (Kratsios et al., 21 Apr 2025).

• Variants Supporting Computational Efficiency: Notable structures include FastKAN, which demonstrates that 3rd-order B-splines used in KANs can be efficiently approximated with Gaussian radial basis functions, resulting in considerable forward and backward speedups (Li, 10 May 2024). Likewise, approaches such as Kolmogorov-Arnold-Fourier Networks combine random Fourier features with hybrid GELU-Fourier activations to better capture high-frequency structure with fewer parameters (Zhang et al., 9 Feb 2025).

3. Function Approximation and Theoretical Guarantees

KANs achieve optimal or near-optimal approximation rates for a wide range of target function classes:

• Dimension-Indifferent Error Bounds: The approximation error for smooth target functions decomposed via KAN, with each univariate function parameterized by splines on a grid of size $G$, satisfies

$\|f - \text{KAN}_G(f)\|_{C^m} \leq C G^{-k-1+m}$

where $k$ is the spline order and $m$ the derivative order (Liu et al., 30 Apr 2024, Basina et al., 15 Nov 2024). This scaling is independent of the input dimension, effectively mitigating the curse of dimensionality.

• Besov Space Optimality: For functions in Besov spaces $B_p^s(\mathcal{X})$, Res-KANs (KANs with skip connections) can achieve optimal rates:

$\|f - \hat f\|_{B_{p,q}^\alpha} < \epsilon$

for any $\alpha < s$, with network width and depth scaling polynomially in $\epsilon$, not in dimension $d$ (Kratsios et al., 21 Apr 2025).

• Sample Complexity: In learning scenarios, KANs enjoy dimension-independent sample complexity bounds under Besov regularity, reflecting efficient learnability in both function values and derivatives.

4. Empirical Performance and Applications

A wide range of studies benchmark KANs against MLPs and other models, demonstrating consistent advantages:

• Parameter Efficiency: For fixed accuracy, KANs require orders of magnitude fewer parameters than MLPs in function regression, PDE solving, and physics-informed tasks (Liu et al., 30 Apr 2024, Faroughi et al., 30 Jul 2025).
• Spectral Representation: KANs better approximate functions with high-frequency, localized, or oscillatory components due to their adaptive basis structure, outperforming both traditional MLPs and fixed-frequency Fourier models (Gleyzer et al., 1 Aug 2025, Zhang et al., 9 Feb 2025).
• Scientific Machine Learning: KANs are applied successfully in data-driven learning (e.g., time series, medical imaging), physics-informed neural networks (PINNs), and deep operator learning (DeepOKANs). Empirical studies report lower mean squared errors, improved convergence, and more stable training dynamics compared to MLPs (Faroughi et al., 30 Jul 2025).
• Symbolic Regression and Interpretability: KANs’ explicit edge functions allow learned models to be converted into interpretable, symbolic formulas, facilitating scientific discovery, model validation, and engineering optimization (Novkin et al., 19 Mar 2025, Liu et al., 30 Apr 2024).
• Uncertainty Quantification: Probabilistic KANs employing Divisive Data Re-sorting (DDR) produce input-dependent predictive distributions, capturing aleatoric uncertainty and distributional multimodality (Polar et al., 2021).

5. Extensions and Specialized Variants

KANs admit a range of extensions addressing advanced modeling and deployment needs:

Variant	Key Feature/Domain	Reference
FastKAN	B-spline ≈ RBF network	(Li, 10 May 2024)
KAF/KAN-Fourier	Fourier & GELU hybrid	(Zhang et al., 9 Feb 2025)
Sinusoidal KAN	Weighted sinusoids	(Gleyzer et al., 1 Aug 2025)
InfinityKAN	Variational/adaptive #bases	(Alesiani et al., 3 Jul 2025)
P1-KAN	Piecewise linear, robust	(Warin, 4 Oct 2024)
Hierarchical KAN	No backprop, stacked LS	(Dudek et al., 30 Jan 2025)
Probabilistic KAN	Input-dependent uncertainty	(Polar et al., 2021)

Applications span transfer learning (KAN as a nonlinear probe), autoencoding and representation learning, quantum-inspired architectures (VQKAN, EVQKAN), and operator learning for scientific PDEs.

6. Incorporation of Symmetry and Group Invariance

Recent developments generalize KAN architectures to encode symmetry constraints crucial for physical modeling:

• Equivariance and Invariance: By restructuring the internal representations to depend only on symmetry-preserving invariants (e.g., inner products for $O(n)$ invariance, or permutation-invariant aggregations), KANs can respect $O(n)$, $E(3)$, $S_n$, $GL(n)$, or Lorentz group symmetries. This allows for accurate and sample-efficient modeling of physical systems, molecular dynamics, and particle interactions (Alesiani et al., 23 Feb 2025).
• Equivariant Outputs: For vector-valued functions, outputs can be structured to reflect the proper transformation laws under group actions, ensuring physical consistency.

7. Limitations and Future Research

Despite significant advances, several challenges and open problems remain:

• Computational Overhead: Fine-grained spline or basis expansions on every edge rapidly increase parameter counts and computational cost relative to standard MLPs, motivating research into kernel approximations, parameter sharing, and hardware-aware implementations (Li, 10 May 2024, Zhang et al., 9 Feb 2025).
• Hyperparameter Tuning: The choice of basis type, grid size, degree, and regularization parameters is critical and often nontrivial. Variational approaches such as InfinityKAN directly integrate basis count optimization with learning to address this (Alesiani et al., 3 Jul 2025).
• Integration and Stability: Incorporating KANs into mainstream deep learning frameworks and achieving stable execution on modern hardware (e.g., GPUs) require nontrivial engineering, as recursive or edge-based operations can be less parallelizable.
• Theory: Although strong theoretical rates are established for many function classes, there remain gaps in understanding the generalization properties, expressiveness in the presence of noise, and optimal architecture depth/width trade-offs.

Ongoing work seeks to address these limitations by developing efficient backbone modules, scalable kernel approximations, hyperparameter auto-tuning methods, and hybrid models that combine KANs with convolutional, transformer, or graph neural architectures (Somvanshi et al., 9 Nov 2024, Faroughi et al., 30 Jul 2025).

---

KANs thus occupy a unique position at the intersection of mathematical representation theory and modern deep learning, providing a principled, adaptive, and interpretable alternative to conventional neural networks for high-dimensional scientific and engineering applications.