Hierarchical RBF-KAN and RBF-SKAN Architectures for Multidimensional Function Approximation and Random Field Learning

Abstract: In this manuscript, we propose and analyze hierarchical Kolmogorov--Arnold neural network architectures employing radial basis functions as activation functions for approximating deterministic functions and random field models. Specifically, we develop a hierarchical radial-basis-function Kolmogorov--Arnold network (hierarchical RBF-KAN) for multidimensional deterministic function approximation and a hierarchical radial-basis-function stochastic Kolmogorov--Arnold network (hierarchical RBF-SKAN) for random field learning. From a theoretical perspective, we establish universal approximation results for both architectures. In particular, we derive quantitative approximation estimates for the hierarchical RBF-KAN, showing that the proposed framework has the potential to partially alleviate the curse of dimensionality in learning high-dimensional functions by reducing the effective dimensionality of the approximation problem. Furthermore, we show that the hierarchical RBF-SKAN can approximate random field models under the Wasserstein-2 metric. Empirically, we show that our proposed radial-basis-function-based neural network structure could effectively learn multivariate functions and random field models.

• The paper introduces novel hierarchical RBF-based architectures leveraging Kolmogorov-Arnold representations to decompose multivariate functions.
• The hierarchical RBF-KAN achieves efficient approximation with error bounds independent of dimension and outperforms standard RBF-MLPs on oscillatory functions.
• The RBF-SKAN extension models random fields via stochastic parameterization, providing robust uncertainty quantification and improved variance predictions.

Hierarchical RBF-KAN and RBF-SKAN Architectures for High-Dimensional Function Approximation and Random Field Learning

Introduction and Motivation

This work presents hierarchical Kolmogorov-Arnold neural network (KAN) architectures leveraging radial basis functions (RBFs) for two central tasks: deterministic high-dimensional function approximation and data-driven learning of random fields. The proposed architectures, termed hierarchical RBF-KAN and hierarchical RBF-SKAN, are designed to provide quantitative improvements in approximation efficiency, expressivity, and uncertainty quantification, in comparison to existing RBF-based neural networks and KAN variants.

A notable theoretical innovation is the explicit hierarchical block structure, inspired by the Kolmogorov-Arnold representation, which decomposes multivariate functions into sums over univariate nonlinear transforms. The use of localized Gaussian RBF nonlinearities provides robust universal approximation guarantees, including quantitative estimates on approximation error and neuron count, with partial mitigation of the curse of dimensionality. The architecture further extends to random field modeling (RBF-SKAN), supporting universal approximation in measure (Wasserstein-2 metric) for diverse stochastic processes.

Hierarchical RBF-KAN: Architecture and Theory

The hierarchical RBF-KAN employs a multi-block design, where each block contains two activation layers with distinct neuron counts, configurated according to Kolmogorov-Arnold's finite-sum representation. RBF activation is implemented using Gaussian kernels with trainable scales and centers, ensuring localized, adaptable nonlinearity. Importantly, the hierarchical structure enforces that the number of neurons in each subsequent activation narrows, facilitating dimensionality reduction in the approximation space.

Figure 1: Structure of the hierarchical RBF-KAN, illustrating both the one-block and multi-block variants, each block comprising two activation layers with Gaussian kernel activations and respecting Kolmogorov-Arnold decomposition constraints.

Theoretical results provide a universal approximation theorem for both scalar and vector-valued multivariate continuous functions on compact domains, with explicit error bounds relating approximation accuracy to neuron count and the RBF scale. Under mild smoothness and boundedness assumptions, the error for a $d$-dimensional target function can be bounded independently of dimension, given appropriate parameterization, thereby partially alleviating the curse of dimensionality. Quantitative analysis establishes that the convergence rate is at most $\mathcal{O}(N^{-1/10})$ for $k \geq 2$, where $N$ is proportional to the number of RBFs per univariate component.

Empirically, the architecture outperforms standard RBF-MLPs, naive (non-hierarchical) RBF-KANs, and multilayer perceptrons with standard activations (ReLU, Tanh, Sigmoid), especially on highly oscillatory and high-dimensional functions. The inclusion of ResNet-style skip connections is critical for both stability and accuracy in deeper hierarchical configurations.

Stochastic Extension: Hierarchical RBF-SKAN

The stochastic version, hierarchical RBF-SKAN, generalizes RBF-KAN by introducing controlled randomness into the network parameters (notably the weights and RBF scales in specific layers). This design targets the learning of random fields: stochastic mappings $\bm{x} \mapsto \bm{y}(\bm{x};\omega)$, where $\omega$ represents latent uncertainty.

The architecture employs two hierarchical RBF-KAN modules in series. The first is deterministic, mapping $\bm{x}$ to a high-dimensional nonlinear latent space. The second module is stochastic, where its weights and scale parameters are sampled from parameterized distributions (e.g., uniform with trainable bounds), introducing output stochasticity conditioned on $\bm{x}$.

Figure 2: Hierarchical RBF-SKAN composed of two hierarchical RBF-KAN modules, with the second module parameterized stochastically. Randomness in network parameters enables modeling of conditional distributions of random fields.

A universal approximation theorem is proved for RBF-SKAN, showing that, under weak regularity assumptions, any target random field model's conditional law can be approximated in Wasserstein-2 distance arbitrarily closely by a suitable RBF-SKAN instantiation. The proof leverages a cell-wise local representation and construction of coupling measures between the model output and the ground-truth field.

Numerical Experiments

Multidimensional Function Approximation

Experiments on oscillatory high-dimensional functions demonstrate that hierarchical RBF-KAN consistently achieves lower test error than RBF-MLPs, standard KANs (using splines), and deep MLP baselines with Tanh, Sigmoid, or ReLU activations. The performance gains are especially pronounced as the input dimension increases, with error deteriorating much more slowly in RBF-KAN relative to all baselines.

Notably, the number of hierarchical blocks and the presence of ResNet structure both substantially enhance accuracy. Replacing the RBF kernel with Tanh nonlinearities in the same architecture is suboptimal for highly oscillatory targets, highlighting the necessity of both structure and nonlinearity.

Chaotic Dynamical System Reconstruction

On learning the Lorenz system, hierarchical RBF-KAN accurately reconstructs state trajectories and dynamics under uncertain initializations, outperforming both multi-layer MLPs and alternative RBF-based models in terms of average relative trajectory and dynamics error.

Figure 3: (a)–(c) depict Lorenz system state predictions—ground truth vs. hierarchical RBF-KAN; (e)–(f) provide relative error comparisons with alternative neural architectures.

Random Field Learning

The hierarchical RBF-SKAN is evaluated on the task of reconstructing random fields with both oscillatory and discontinuous dependence on $\bm{x}$. It is compared to two prevalent uncertainty quantification frameworks: conditional variational autoencoders (CVAE) and conditional normalizing flows (CNF), using both standard (GELU) and RBF activations.

Key numerical findings:

• RBF-SKAN achieves the lowest error in reconstructing standard deviation across increasing input dimensions; CVAE exhibits large variance errors that degrade with dimensionality.
• RBF-SKAN and CNF perform similarly on mean prediction error, while RBF-SKAN is much more robust for variance prediction.
• Introducing stochasticity in the scale parameters (beyond the latent variables) further improves variance reconstruction.
• Computational cost of RBF-SKAN is higher (due to Wasserstein-based loss), however, it is justified by significantly improved predictive uncertainty estimates.

Implications and Future Directions

The hierarchical RBF-KAN and RBF-SKAN architectures advance the state of the art in both deterministic and stochastic high-dimensional approximation. Theoretically, they provide constructive, structure-aware universality results that elucidate the interplay between architecture design and expressivity, placing specific focus on alleviating dimensionality bottlenecks linked to classical shallow RBF networks.

Practically, these methods enable more reliable and interpretable surrogate modeling, especially in realms such as physics-informed learning, uncertainty quantification, and scientific simulation, where both sample efficiency and accuracy in high dimensions are crucial. The random field modeling framework via RBF-SKAN is directly applicable to probabilistic surrogate modeling and Bayesian inference workflows.

Potential avenues for future research include:

• Deeper hierarchies: Scaling the number of blocks and layers to further improve approximation in even higher dimensions.
• Integration as modular blocks within Transformer or operator learning architectures.
• Extension to discrete-output and classification tasks by reconfiguring activation and loss structures.
• Reduction of Wasserstein-based training cost, e.g., via entropy-regularized Sinkhorn approximations.
• Rigorous exploration of optimal parameterization (number of RBFs, neuron counts per layer) for specific function classes.

Conclusion

Hierarchical RBF-KAN and RBF-SKAN architectures provide a principled, theoretically grounded, and empirically effective framework for high-dimensional function approximation and random field learning. Their hierarchical design, grounded in Kolmogorov-Arnold theory, yields significant expressivity gains, robustness to dimensionality, and superior uncertainty quantification compared to standard neural strategies. These properties make them a valuable addition to the toolbox for researchers working on scientific machine learning, surrogate modeling, and probabilistic learning in high dimensions.