SparseRBFnet: Sparse RBF Network

• SparseRBFnet is a shallow network that uses adaptive, ℓ1-regularized radial basis functions to produce sparse yet accurate function approximations.
• It employs a multi-phase training protocol—including kernel insertion, semi-smooth second-order optimization, and pruning—to optimize centers, widths, and coefficients.
• Applications span meshfree PDE solvers, molecular modeling, and geometric reconstruction, achieving high accuracy with significantly fewer kernels.

Sparse Radial Basis Function Networks (SparseRBFnet) are shallow kernel-based neural architectures employing adaptive, sparse superpositions of radial basis functions (RBFs) for function approximation, PDE solving, and geometric and scientific modeling. Their key distinguishing feature is the imposition of sparsity—primarily via $\ell_1$ regularization or total variation penalties—on expansion coefficients, yielding highly compact representations that outperform classical mesh-based methods and fully dense kernel networks across a variety of domains including numerical PDEs, molecular modeling, and geometric reconstruction (Shao et al., 12 May 2025, Shao et al., 24 Jan 2026, Wang et al., 2023, Lian et al., 5 May 2025, Gui et al., 2020).

1. Network Formulation and Function Space Foundations

SparseRBFnet selects a representation of the target function $u(x)$ as a finite sum over $N$ radial basis units:

$$u(x) = \sum_{n=1}^N c_n\, \phi\bigl((x-y_n)/\sigma_n\bigr)$$

where $c_n \in \mathbb R$ are coefficients, $y_n$ are centers, $\sigma_n > 0$ are adjustable widths, and $\phi: \mathbb R^d \to \mathbb R$ is a radial kernel, typically Gaussian or Matérn. In anisotropic and geometric variants, $\sigma_n$ may be replaced with ellipsoid shape matrices $D_n$ and rotations $\Theta_n$ or $R_n$ (Lian et al., 5 May 2025, Gui et al., 2020).

SparseRBFnet's theoretical foundation lies in the Reproducing Kernel Banach Space (RKBS) induced by measures over center/width pairs, with total variation norm:

$$u(x) = \int_\Omega \phi(x; \omega) d\mu(\omega), \quad \Omega = \{(y,\sigma): y \in \mathbb R^d, 0 < \sigma \leq \sigma_{\mathrm{max}}\}$$

The main result is that, under mild regularity and integrability conditions on $\phi$, the infinite-width closure of such networks coincides with the atomic norm and is isomorphic to a Besov space $B^s_{1,1}(D)$ almost independently of kernel choice (Shao et al., 12 May 2025, Shao et al., 24 Jan 2026). The bounded variation on coefficients promotes sparsity, yielding network expansions with far fewer terms than classical collocation or dense Gaussian process methods.

2. Loss Functions, Regularization, and Optimization Strategies

Training a SparseRBFnet typically involves minimization of an empirical (collocation-based) loss functional augmented by a sparsity-promoting penalty. For PDEs:

$$L(u) = \frac{1}{2} \sum_{i=1}^{K_1} w_i |E[u](x_i)|^2 + \frac{\lambda}{2} \sum_{j=1}^{K_2} w_j' |B[u](x_j')|^2 + \alpha\, \|c\|_1$$

where $E$ and $B$ are the PDE and boundary residual operators, $\|c\|_1$ is the $\ell_1$ norm (total variation penalty), and $\alpha$ tunes sparsity (Wang et al., 2023, Shao et al., 12 May 2025, Shao et al., 24 Jan 2026, Gui et al., 2020).

The empirical success of SparseRBFnet is contingent on a multi-phase training protocol:

• Phase I (Feature Insertion/Boosting): Greedily select new kernel candidates based on directional derivative (“dual certificate”) maximizing loss descent.
• Phase II (Semi-smooth Second-order Optimization): Adapt both coefficients and “inner weights” (centers, widths) using Newton or Levenberg–Marquardt-like updates, essential for preventing kernel clustering and achieving efficient sparsification.
• Phase III (Pruning): Drop kernels with coefficients driven to zero under the $\ell_1$ regularization regime.

This three-phase procedure contrasts with traditional RBF networks, which typically optimize all parameters simultaneously using first-order methods and do not incorporate adaptivity nor aggressive pruning (Shao et al., 12 May 2025, Shao et al., 24 Jan 2026, Wang et al., 2023, Gui et al., 2020).

3. Network Extensions: Anisotropy, Multiscale, and Hierarchical Adaptation

Anisotropic and ellipsoid RBF neurons generalize the radial kernel via:

$$\phi_j(x) = \exp\left(-\frac{1}{2}(x-\mu_j)^T \Sigma_j^{-1} (x-\mu_j)\right) = \exp[- \|D_j^{1/2} \Theta_j (x-\mu_j)\|_2^2]$$

where $D_j$ is a principal-axis diagonal matrix and $\Theta_j$ an orthogonal rotation (parameterized by Euler angles) (Lian et al., 5 May 2025, Gui et al., 2020). Such parameterizations allow more accurate encoding of geometric or tensorial structure (e.g., molecular densities, signed distance functions) with fewer kernels.

For multiscale problems, especially elliptic PDEs with rapidly oscillating coefficients, SparseRBFnet exploits shape parameter initialization $D_i$ in $[0, 1/\varepsilon]$ and achieves sparse scaling:

$$N(\varepsilon) = O(\varepsilon^{-n\tau}), \quad 0<\tau<1$$

notably beating mesh-based methods which require $O(\varepsilon^{-n})$ degrees of freedom (Wang et al., 2023).

Hierarchical octree refinement accelerates training and improves convergence by sequentially fitting from coarse to fine lattice point sets, leveraging transfer learning and local support (Lian et al., 5 May 2025).

4. Theoretical Guarantees and Function Space Characterization

SparseRBFnet admits a representer theorem: the minimizer to the infinite-dimensional (measure) sparse optimization problem necessarily has finite, Dirac-supported representation,

$\mu^* = \sum_{n=1}^{N} c_n \delta_{\omega_n}$

with $N \leq \bar{N}$ (the total number of collocation constraints). This finite expansion ensures computational tractability and justifies adaptive kernel selection (Shao et al., 12 May 2025).

Error bounds under well-posedness assumptions guarantee, for sufficiently dense collocation and suitable choice of $\alpha$,

$$\|u_{\alpha} - u\|_{U} \leq C \sqrt{ \|u-u^*\|_{U_0}^2 + \alpha \|\mu^*\|_{M(\Omega)} }$$

where $u$ is the true solution, $u^*$ the closest network approximant, and $C$ a domain- and operator-dependent constant (Shao et al., 12 May 2025).

Under mild regularity, the solution set imbued by the union of all admissible RBF kernels is a Besov space $B^s_{1,1}(D)$, independent of the specific kernel form. Empirically, kernel choice (e.g., Gaussian, Matérn) affects little beyond per-kernel cost and smoothness tuning, except in highly anisotropic or low regularity settings (Shao et al., 24 Jan 2026).

5. Applications in Scientific Computing, Geometry, and Molecular Modeling

SparseRBFnet has demonstrated high accuracy and compactness across multiple scientific domains:

• Numerical PDEs: Efficient mesh-free solvers for nonlinear, high-order, and fractional PDEs (e.g., bi-Laplacian, fractional Poisson), with adaptively $O(10$–$10^3)$ kernels and sub-percent $L^2$ errors (Shao et al., 12 May 2025, Shao et al., 24 Jan 2026, Wang et al., 2023).
• Molecular Modeling: Sparse ellipsoid networks reconstruct molecular Gaussian densities from atomic coordinates with $<10\%$ as many kernels as atoms. Surface and volume errors below $2\%$ and Hausdorff distance $<0.8$\AA{} are achieved, enabling coarse-grained modeling (Gui et al., 2020).
• Point Cloud Geometry: SE-RBFNet uses joint optimization of weights, centers, shapes, and rotations with dynamic multi-objective regularization and hierarchical octree training to approximate signed distance functions at high fidelity and low kernel count. Empirical evaluation shows significant speedup and accuracy gains relative to previous sparse methods, e.g. on Thingi10K or ABC datasets (Lian et al., 5 May 2025).
• Multiscale PDEs: SparseRBFnet realizes robust performance where classical mesh-based approaches become intractable (e.g., scale separation, discontinuity), maintaining accuracy with $N(\varepsilon) = O(\varepsilon^{-n\tau})$ scaling and outperforming PINN, Deep Ritz, and Deep Galerkin in error metrics (Wang et al., 2023).

6. Operator Calculus and Computational Considerations

For integer-order differential operators, SparseRBFnet enables near-closed-form evaluation via chain-rule differentiation of radial kernels:

$$\nabla_x \phi(\rho) = \phi'(\rho)/\sigma \cdot e, \quad e=(x-y)/|x-y|$$

Fractional Laplacians $(–\Delta)^{\beta/2}$ are computed quasi-analytically via Hankel-transform representation:

$$(–\Delta)^{\beta/2}\phi(|x–y|/\sigma) = \sigma^{–\beta}K_d^{\beta/2}(|x–y|/\sigma)$$

where $K_d^{\beta/2}$ is a one-dimensional integral over the Hankel transform of $\phi$. This structure allows meshfree evaluation of both local and nonlocal operators within collocation frameworks (Shao et al., 24 Jan 2026).

Computational cost per iteration scales linearly in the number of active kernels. Parallelization strategies (CUDA, nearest-neighbor filtering, spatial indices) are employed to speed training, especially in geometric reconstruction applications (Lian et al., 5 May 2025). Octree refinement and adaptive training further mitigate local minima and accelerate convergence.

7. Empirical Findings, Limitations, and Practical Guidance

SparseRBFnet demonstrates consistent numerical advantages over Gaussian process and PINN-type methods, primarily by:

• Automatic, per-kernel adaptation of centers and widths—eliminating the need for manual scale tuning.
• Rigorous representer theorem and error analysis in RKBS/Besov settings.
• Second-order optimization and adaptive insertion, ensuring rapid and robust convergence with minimal overfitting artifacts.
• Compactness and scalability to moderate ambient dimensions ($d\leq4$), with kernel count much smaller than classical mesh sizes required for analogous accuracy.

Limitations include computational scalability to very high $d$, sensitivity to kernel parameterization in extremely anisotropic or discontinuous problems, and the need for careful regularization to avoid premature pruning. For meshfree PDEs, resolution limitations echo those of finite-difference methods, where accuracy saturates or declines when problem scales exceed kernel coverage (Shao et al., 24 Jan 2026, Shao et al., 12 May 2025).

Recommended practice involves initializing the kernel count above the expected final sparsity level, using robust collocation schemes, and activating the $\ell_1$ regularization only after $L_2$ residuals have substantially decreased. Full adaptivity (centers, widths, coefficients) produces the sparsest, most accurate solutions but at higher computational cost compared to outer-only optimization.

---

SparseRBFnet constitutes a unifying, theoretically grounded framework for compact function approximation and meshfree PDE solving, validated across geometric, molecular, and scientific computing applications. The RKBS/Besov characterization, representer theorem, adaptive training protocol, and empirical evidence collectively support its role as an effective and versatile tool for high-accuracy, sparse solution construction in challenging domains (Shao et al., 12 May 2025, Shao et al., 24 Jan 2026, Wang et al., 2023, Lian et al., 5 May 2025, Gui et al., 2020).