Adaptive RBF-KAN: A Comparative Evaluation of Dynamic Shape Parameters in Kolmogorov-Arnold Networks

Abstract: Kolmogorov-Arnold Networks (KANs) approximate multivariate functions using learnable univariate edge functions, typically parameterized by B-spline bases. Although effective, spline-based implementations can be computationally expensive. A modified version of KANs, called FastKAN, improves efficiency by replacing splines with Gaussian radial basis functions (RBFs), but it relies on a fixed kernel and shape parameter. In this work, we extend the RBF-based KAN framework by introducing a broader family of radial basis kernels and by initializing the kernel shape parameter using leave-one-out cross-validation (LOOCV). To the best of our knowledge, this is the first study that integrates LOOCV-based kernel scale estimation with deep KAN training. We also introduce Matérn and Wendland kernels into the KAN framework for the first time, enabling more flexible basis representations beyond the Gaussian kernel used in FastKAN. The LOOCV estimate provides a data-driven initialization of the kernel scale, which is subsequently refined during network training. The proposed adaptive RBF-KAN is evaluated on several two-dimensional benchmark functions. The results highlight the importance of kernel selection and adaptive shape parameters, with different kernels showing advantages for smooth functions, discontinuities, and oscillatory patterns. Overall, combining LOOCV-based initialization with adaptive kernel learning provides a practical strategy for improving RBF-based KAN models.

• The paper introduces an adaptive RBF-KAN model that integrates LOOCV-based initialization and gradient-driven shape parameter tuning to enhance function approximation.
• The methodology demonstrates improved accuracy across diverse function classes including smooth, discontinuous, oscillatory, and singular targets, with significant error reduction metrics.
• Expanding the RBF kernel family with Matérn and Wendland functions, the study provides a flexible neural architecture bridging classical kernel methods and deep networks.

Adaptive RBF-KAN: Dynamic Shape Parameterization in Kolmogorov-Arnold Networks

Introduction

This paper presents a comprehensive evaluation of dynamic shape parameterization in Kolmogorov-Arnold Networks (KANs), specifically extending the RBF-based FastKAN variant by introducing adaptive kernel learning and the systematic integration of diverse radial basis kernels beyond the traditional Gaussian form. The innovations include leveraging leave-one-out cross-validation (LOOCV) for data-driven initialization of kernel bandwidths and learning these shape parameters during training. It further introduces Matérn and Wendland kernels within the KAN framework, exploring how kernel selection and shape adaptation enable greater modeling flexibility across function classes.

Background and Positioning

KANs are motivated by the Kolmogorov-Arnold representation theorem, suggesting that any continuous multivariate function can be represented as finite compositions and sums of univariate continuous functions. Unlike MLPs, which impose fixed activation functions per neuron, KANs learn univariate functions on the edges between neurons, originally parameterized as B-splines for local control and strong approximation capabilities. However, the computational overhead of spline evaluation has driven the development of variants such as FastKAN (Gaussian RBF replacements for splines), EfficientKAN (memory efficiency via matrix multiplication reformulations), and several others employing alternative basis families.

A major limitation noted in prior RBF-based KANs is the reliance on fixed kernel shape parameters ($h$), a critical hyperparameter controlling the locality and smoothness of the basis functions. Fixed $h$ cannot account for the heterogeneity in structural complexity across different target functions—leading either to overfitting or excessive smoothing. Furthermore, existing implementations predominantly use the Gaussian RBF, limiting modeling diversity. This paper addresses these two issues systematically: adaptive kernel learning and an expanded RBF kernel family.

Methodology

The adaptive RBF-KAN architecture generalizes edge functions as linear expansions over arbitrarily chosen RBF kernels, parameterized by center grids and a shape parameter $h$:

$$\psi_{k,m,n}(x) = \sum_{j=1}^{K} c_{k,m,n,j} \, \phi\left(\frac{|x-c_j|}{h}\right)$$

LOOCV-Based Initialization

Selection of $h$ is critical for the performance of kernel methods. The paper employs LOOCV error minimization (utilizing Rippa's closed-form for error computation) on auxiliary one-dimensional kernel expansions to obtain a data-adaptive initialization, greatly reducing the need for manual or exhaustive search over bandwidths. This approach is shown to be robust, computationally efficient, and effective as a pre-training initialization strategy.

Dynamic Shape Parameter Learning

While LOOCV provides a strong static initialization, the model further adapts the shape parameter during training using gradient-based optimization. By parameterizing $h = \exp(\theta)$, positivity is enforced throughout optimization, with both spline/RBF coefficients and $h$ jointly updated to minimize the supervised loss. This dynamic adaptation enables the RBF representation to better conform to varying local smoothness or discontinuity in the data.

Kernel Family Generalization

The approach encompasses a broader RBF kernel family; specifically, Matérn and Wendland kernels with variable degrees of smoothness are included. These choices allow fine-grained tradeoffs between global and local approximation, and their basis support and smoothness adapt to distinct target function characteristics.

Experimental Evaluation

Benchmarking is performed on four canonical bivariate function classes: the smooth Franke function, a function with a circular discontinuity, a high-frequency oscillatory function, and a function with a localized singularity. The evaluation rigorously aligns training regimes for all models and compares adaptive RBF-KAN against MLPs, spline-based KANs, FastKAN (fixed $h$), EfficientKAN, Chebyshev KAN, KAF, and others.

Numerical Results

• Smooth Targets: For the Franke function, the Gaussian kernel with adaptive $h$ achieves relative $L_2$ errors of $h$0, outperforming all fixed-shape RBFK approaches and matching or surpassing spline-based KANs, while requiring far less computational time than B-spline implementations.
• Discontinuities: For circular discontinuities, Matérn kernels (especially M2) provide an order-of-magnitude improvement in error over Gaussian RBFs, with adaptive $h$1 consistently yielding lower $h$2 loss.
• Oscillatory/High-Frequency: Compactly supported Wendland kernels (notably W2) dramatically enhance high-frequency data approximation; the adaptive approach reduces error from $h$3 (fixed Gaussian) to $h$4.
• Singularity: For localized peaks, high-order Wendland kernels (W6) halve the error compared to Gaussian baselines, indicating superior capacity for singularity tracking.

The empirical findings robustly support the claim that adaptive kernel selection and data-driven initialization provide significant and robust improvements in accuracy and convergence across diverse function classes.

Comparative Analysis

• Spline-based KANs deliver high accuracy but incur prohibitive computational overhead.
• MLPs, while efficient, underperform in complex or high-frequency settings due to activation function rigidity.
• Other KAN variants (EfficientKAN, KAF, Chebyshev) display mixed performance and seldom match the overall balance of efficiency and accuracy seen in adaptive RBF-KAN.
• The adaptive approach consistently matches or outperforms all baselines for all function types tested.

Theoretical and Practical Implications

The integration of LOOCV-driven bandwidth selection with KAN architectures provides a principled bridge between classical kernel methods and deep compositional networks, highlighting important connections in universal approximation theory. Additionally, the ability to dynamically adapt kernel shape parameters as part of end-to-end training addresses longstanding issues in kernel-based neural modeling, particularly with regard to overfitting, generalization, and automatization of parameter selection.

By expanding the permissible kernel basis to include Matérn and Wendland functions, this work demonstrates that basis family selection is crucial for optimally fitting different structural regimes in data. It thus points toward a meta-learning paradigm for basis selection in deep networks, where both basis type and internal kernel parameters are co-optimized.

Future Directions

• Scalability: Extending adaptive RBF-KAN to high-dimensional function approximation, where grid-based center placements and kernel support selection may become computational bottlenecks, is a key area.
• Noisy and Real-World Data: The impact of adaptive kernel learning on overfitting and robustness under noise, and its integration with explicit regularization strategies, should be investigated.
• Scientific Computing Applications: The architecture’s suitability for PDE solution, scattered data interpolation, and uncertainty quantification invites further interdisciplinary application.
• Automated Basis Selection: Future frameworks could incorporate meta-learning or Bayesian optimization over the kernel family, potentially blending kernel types within a single architecture.

Conclusion

This paper establishes that KANs equipped with adaptive RBF-based edge parameterization, guided by LOOCV initialization and enriched kernel families, surpass prior variants in both empirical performance and modeling flexibility across data types. The systematic demonstration of kernel and shape parameter impacts, alongside reproducible experimental comparisons, underscores the critical importance of adaptive, data-driven kernel learning in neural architectures inspired by the Kolmogorov-Arnold framework. The results support the use of adaptive RBF-KAN as a highly competitive and flexible function approximator, particularly in scientific computing contexts where approximation accuracy and computational efficiency are both paramount.

---

Reference:

"Adaptive RBF-KAN: A Comparative Evaluation of Dynamic Shape Parameters in Kolmogorov-Arnold Networks" (2605.21534)