Adaptive Kernel RBF Framework

• The paper demonstrates that adaptive kernel RBF frameworks improve model performance by allowing data-driven tuning of hyperparameters instead of using fixed values.
• The methodology employs locally adaptive bandwidths, multi-kernel fusions, and meta-parameter optimization to achieve state-of-the-art results in regression and classification tasks.
• Empirical results show significant accuracy improvements and reduced support sizes compared to conventional fixed-kernel methods, validating the framework's robustness.

An adaptive kernel Radial Basis Function (RBF) framework is defined by the property that one or more kernel hyperparameters (such as bandwidth, shape, or metric) are trainable or data-dependent, which contrasts with traditional RBF approaches employing globally fixed kernel parameters. Adaptive kernel RBF frameworks have emerged across kernel machines, neural models, and numerical methods as a response to the expressivity, scalability, and robustness limitations of fixed kernels. These frameworks include locally adaptive bandwidths, mixtures of multiple kernel types, meta-parameter gradients, probabilistic Bayesian updates, and data-driven, context-specific shape adjustments. This article presents the main technical underpinnings, representative algorithmic realizations, and a survey of methods and results from state-of-the-art research, with detailed references to key developments in the literature.

1. Locally Adaptive Bandwidths and Asymmetric Kernels

Traditional Gaussian RBF kernels use a fixed global scale $\sigma$, resulting in limited flexibility:

$$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$

The Locally-Adaptive-Bandwidth (LAB) RBF kernel generalizes this by attaching a vector of positive scales $\theta_i \in \mathbb{R}_+^m$ to each support point $x_i$, yielding

$$K(x_i, x) = \exp\left(-\|\theta_i \odot (x-x_i)\|_2^2\right),$$

where $\odot$ is the Hadamard product. This formulation is asymmetric ($K(x_i, x_j) \neq K(x_j, x_i)$), dramatically increasing kernel expressivity and enabling adaptation to local data heterogeneity. The LAB–RBF framework introduces a new asymmetric kernel ridge regression (KRR) formulation, utilizing dual feature maps $\varphi,\psi$ and an objective with a negative cross-term that reflects the non-symmetry. The learning procedure alternates between solving KRR for fixed bandwidths and adapting the bandwidths on support points via stochastic gradient descent over a large training batch. Dynamic support selection is employed, where support points are iteratively added based on residual errors until an error or budget threshold is reached.

Empirically, LAB–RBF achieves significantly higher $R^2$ compared to global RBF–KRR, multiple kernel learning, and even state-of-the-art neural networks (e.g., ResNet, wide neural nets). On the Airfoil dataset, LAB–RBF achieves $R^2 = 0.961$, compared to $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$0 (RBF–KRR) and $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$1 (Falkon), using only $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$2200 supports (vs. 900 for Falkon); on SML, $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$3 (LAB–RBF), compared to $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$4 (Falkon, $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$52,000 centers) and $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$6 (RBF–KRR) (He et al., 2023).

2. Adaptive and Multi-Kernel Compositions in Neural Architectures

RBF neural networks (RBFNNs) are a classical paradigm where the hidden layer comprises $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$7 RBF units. Contemporary approaches exploit adaptive kernel compositions for each hidden unit. In multi-kernel RBFNN (“Co-RBFNN”), each unit forms a locally weighted fusion of $$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$8 base kernels, e.g., Gaussian and cosine:

$$K(x, x') = \exp\left(-\frac{\|x-x'\|^2}{\sigma^2}\right).$$9

with adaptive weights $\theta_i \in \mathbb{R}_+^m$0. These parameters are updated per-unit via stochastic gradient descent to minimize squared error, yielding improved convergence, reduced steady-state error, and greater resilience to poor local minima compared to models with fixed or globally mixed kernels (Atif et al., 2020). Empirical results demonstrate superior classification accuracy and rapid convergence on established tasks.

Another mechanistic avenue is adaptive kernel fusion, such as joint Euclidean and cosine kernels with trainable convex weights, updated via gradient descent to minimize the prediction error (Khan et al., 2019). This yields performance that always matches or outperforms the best single kernel or manual fusion.

3. Meta-Parameter Optimization, Greedy Schemes, and Analytic Selection

Many adaptive kernel RBF methods optimize meta-parameters using data-specific objectives rather than manual grid search:

• Cluster-based RBF approaches (CkRBF) partition the data space using $\theta_i \in \mathbb{R}_+^m$1-means, then compute local covariance matrices $\theta_i \in \mathbb{R}_+^m$2 per cluster and define the kernel as $\theta_i \in \mathbb{R}_+^m$3. This locality robustly adapts kernel anisotropy and increases grid-search stability for the width hyperparameter (Czarnecki et al., 2014).
• Analytic approaches derive closed-form formulas for kernel parameters. For example, minimizing intraclass diameter and maximizing interclass separation in the feature space produces $\theta_i \in \mathbb{R}_+^m$4, where $\theta_i \in \mathbb{R}_+^m$5 and $\theta_i \in \mathbb{R}_+^m$6 are the maximum within-class diameter and minimum interclass distance, respectively. This removes the need for computationally expensive cross-validation or grid search and preserves accuracy within 1–2% of fully tuned SVMs on benchmarks (Remaki, 1 Apr 2026).
• Two-layered RBF kernel machines utilize a full linear mapping $\theta_i \in \mathbb{R}_+^m$7 as a first "layer" before a classical radial kernel, with $\theta_i \in \mathbb{R}_+^m$8 optimized by minibatch cross-validation, allowing adaptation to the data’s eigenstructure and compressing inactive subspaces (Wenzel et al., 2023).

4. Adaptive RBFs in Deep and Structured Networks

Kolmogorov–Arnold Networks (KANs), which use learnable univariate edge functions, can be parameterized by different RBF kernels instead of B-splines. Adaptive RBF-KAN variants encompass:

• Free-RBF-KAN (Chiu et al., 12 Jan 2026): Each univariate basis combines trainable centroids and smoothness parameters per grid point. All parameters are adapted jointly by gradient-based optimization, enabling flexible, efficient, and expressive function approximation. The architecture maintains the superposition structure of KAN and achieves universality on $\theta_i \in \mathbb{R}_+^m$9. Empirical results show accuracy comparable to B-spline KAN with faster convergence, especially in high-dimensional, structured settings.
• Adaptive RBF-KAN with LOOCV initialization (Cavoretto et al., 20 May 2026): Extends FastKAN by initializing the kernel scale via leave-one-out cross-validation (LOOCV) and allowing the parameter to adapt during deep network training. Various kernel families (Gaussian, Matérn, Wendland) provide further adaptivity to smoothness, oscillatory, or discontinuous behaviors in the function.

Both approaches demonstrate that data-dependent, learnable basis width and type have substantial impact on sample efficiency and function recovery error, especially for non-smooth target functions.

5. Adaptive RBF Interpolation and PDE Solvers

Adaptive RBFs play a fundamental role in mesh-free numerical solution of partial differential equations (PDEs):

• For multidimensional spatiotemporal integrodifferential equations, RBF expansions with time-dependent coefficients, centers, and anisotropic scales allow automatic adaptation to spatial anisotropy and transients. The parameters are controlled via a neural ODE trained to minimize an operator mismatch loss including PDE residuals and initial/boundary data. Theoretical analysis shows that local scale adaptation partially mitigates the curse of dimensionality in anisotropic domains, while experiments confirm improvements—by 1–2 orders of magnitude in $x_i$0-error—over non-adaptive RBF baselines (Xia et al., 7 Apr 2026).
• Data-dependent RBF interpolation modifies the local shape parameter $x_i$1 at each center $x_i$2, driven by local smoothness indicators (e.g., squared second differences or Laplacians). In discontinuous regions, $x_i$3, producing delta-like kernels that suppress Gibbs phenomena. The resulting adaptive interpolation matrix remains invertible and avoids ill-conditioning. Error and condition number estimates are essentially unchanged in smooth regions, while oscillations are strongly reduced near jumps (Kuruc et al., 24 Mar 2026).

6. Online, Probabilistic, and Feature-Based Adaptation

Stream-based or sequentially updated adaptive kernels have been developed for signal processing and time-series:

• Adaptive Random Fourier Features Kernel LMS (ARFF-GKLMS) (Gao et al., 2022) directly adapts both the RFF weights and the underlying frequency vectors/phases representing the RBF kernel, thus learning the kernel bandwidth online as part of a stochastic gradient procedure. This achieves better steady-state error and faster convergence than fixed bandwidth or classical growing-dictionary kernel methods, especially under nonstationarity.
• Probabilistic KAFs (Castro et al., 2017) use a Bayesian formulation with priors on kernel parameters, dictionaries, and weights, learning all components—including RBF bandwidth—jointly (via MAP or MCMC) with dedicated priors to induce sparsity and enforce desired properties. The online scheme employs sliding window parameter inference to remain fully adaptive, significantly reducing MSE and dictionary redundancy relative to standard approaches.
• Deterministic feature map methods (Li et al., 2019) construct finite-dimensional, polynomial-exact explicit feature mappings for RBFs (using, e.g., Gaussian quadrature or Taylor expansion), yielding a positive definite kernel and a finite-dimensional RKHS. Online algorithms operating entirely in this space—without the kernel trick—achieve faster and more robust convergence than random Fourier features.

7. Applications and Empirical Impact

Adaptive kernel RBF frameworks have demonstrated state-of-the-art empirical results across regression, classification, operator learning, PDE solution, and few-shot adaptation:

• In vision-language one-shot adaptation, ReHARK (Islam, 12 Mar 2026) uses an ensemble of multi-scale RBF kernels, hybrid prior construction, support set augmentation, and adaptive distribution rectification. The method leverages global KRR with proximal regularization, multi-kernel ensembling, and extensive hyperparameter search, achieving one-shot accuracy (65.83%) that surpasses previous methods by 2–7 percentage points on 11 benchmarks.
• Support vector machines (SVMs) have benefited from both per-cluster local adaptivity (Czarnecki et al., 2014), and automated, joint (or analytic) kernel-width selection (Thurnhofer-Hemsi et al., 2020, Remaki, 1 Apr 2026), improving both stability and performance versus grid-tuned global RBF-SVMs.

These results collectively establish that adaptive kernel RBF frameworks alleviate the rigidity of fixed-kernel methods, allowing for superior fit to data heterogeneity, improved generalization, robustness to hyperparameter misspecification, and computational efficiency via judicious parameterization.