---
title: Radial Basis Function Residual LS (RBF–RLS)
url: https://www.emergentmind.com/topics/radial-basis-function-residual-least-squares-rbf-rls
type: topic
---

# Radial Basis Function Residual LS (RBF–RLS)

Radial Basis Function Residual Least Squares (RBF–RLS) is a class of mesh-free numerical approximation and regression methodologies involving radial basis expansions whose coefficients are determined by minimizing the residuals of functional conditions in a discrete least-squares sense. It encompasses both global and localized approaches for high-accuracy function approximation, PDE solution, and regression, providing a framework with well-founded stability and regularization properties, efficient numerical implementations, and deep connections to probabilistic and machine-learning models such as Bayesian RBF networks and LS-SVR.

## 1. Mathematical Formulation

RBF–RLS uses a linear combination of radial basis functions (typically Gaussians) to approximate a target function or solve a PDE. The standard global ansatz over a bounded domain $\Omega\subset\mathbb R^d$ is
\[
u(x) = \sum_{j=1}^N c_j\,\phi(\|x-\xi_j\|),
\]
where $\phi(r)=\exp(-(\epsilon r)^2)$ is the Gaussian basis with shape parameter $\epsilon > 0$, and $\{\xi_j\}_{j=1}^N$ are the centers, often distributed both inside $\Omega$ and in a thin layer outside to mitigate edge effects and ensure coverage [2211.12598]. 

Given $M\gg N$ collocation or residual points $\{x_i\}_{i=1}^M$, the residuals are defined via linear functionals $L_i$, which can encode pointwise evaluation $L_i[u]=u(x_i)$ (approximation) or differential operators $L_i[u]=\mathcal L u(x_i)$ (PDEs). The least-squares system is constructed as
\[
A_{ij} = L_i[\phi(\|x-\xi_j\|)]|_{x=x_i},\quad b_i = g_i,
\]
and the coefficients $c=(c_1,\ldots,c_N)$ solve
\[
\min_{c\in\mathbb R^N} \|A c - b\|_2.
\]
Associated normal equations $(A^T A) c = A^T b$ are solved using a truncated SVD or thresholded QR to regularize high condition numbers inherent to global RBF systems [2211.12598, 2308.14490].

PDE boundary conditions are enforced by stacking additional boundary functionals, yielding an augmented overdetermined system that includes both domain and boundary residuals [2211.12598, 1702.07148]. 

Localized and partition of unity variants employ overlapping subdomains with compactly supported RBFs, building global approximants from local least-squares fits weighted by Shepard-type windows [1702.07148].

## 2. Theoretical Properties and Stability Analysis

The accuracy and stability of RBF–RLS depend critically on the choice of RBF shape parameter $\epsilon$, center placement, and oversampling ratio $\beta = M/N$. In the global case, stability and machine-precision accuracy are achieved by using:
- Linear scaling of the shape parameter with the number of degrees of freedom: $\epsilon = \alpha N^{1/d}$. This yields constant overlap of neighboring Gaussians and controls the coefficient norm growth even as $N\rightarrow\infty$ [2211.12598].
- Placement of centers both inside and outside the domain, forming a thin extension layer to prevent 'edge stagnation' and ensure error estimates that do not degrade near boundaries [2211.12598].
- Oversampling ratio $\beta \gtrsim 2$, ensuring the discrete residual $\ell_2$ norm approximates the continuous $L^2$ norm and rendering the system matrix a well-conditioned frame [2211.12598, 2308.14490, 1702.07148].

Error bounds are established using frame theory; in 1D for Gaussians, sublinear scaling of $\epsilon$ with $N$ yields vanishing error as $N\to\infty$, while linear scaling leads to a reachable machine-precision plateau determined by the SVD cutoff $\tau$ [2211.12598]. In partition-of-unity and local RBF–FD settings, discrete least squares enables high-order convergence and uniform stability (coercivity of the discrete bilinear form), which classical collocation schemes may lack, especially in the presence of Neumann boundaries [2003.03132, 1702.07148].

## 3. Efficient Algorithms and Computational Complexity

Recent advances provide efficient solvers for the overdetermined RBF–RLS systems, especially when the RBF centers are placed on regular grids in a bounding box:
- The matrix $A$ naturally splits into a 'regular' (circulant or block-circulant) part, admitting acceleration via FFTs, and a low-rank correction reflecting boundary and geometry effects [2308.14490].
- The AZ algorithm for RBF–RLS exploits this splitting: it applies a fast solution for the regular part and corrects using a direct small-rank solver for the remainder. In 1D, this leads to overall $\mathcal O(N\log N)$ cost, with the rank of the low-rank correction bounded independently of $N$. In higher dimensions, the FFT cost grows to $\mathcal O(N^2)$ but still yields speedup vs. dense QR solvers [2308.14490].

Practical algorithms typically involve:
- Constructing RBF expansion matrices and right-hand sides at oversampled interior and boundary points,
- Regularizing via SVD or QR with an absolute tolerance,
- Evaluating the approximant on arbitrary points via the reconstructed coefficients [2211.12598, 2308.14490].

Partition-of-unity RBF–RLS achieves further efficiency by reusing factorizations across patches and allows control of the stability norm via oversampling, enabling 5–10× speedup over direct collocation at a given accuracy [1702.07148].

## 4. Extensions: Local and Partition-of-Unity RBF–RLS

RBF–RLS generalizes smoothly to several key frameworks:
- Local RBF–FD (finite difference) methods replace global expansions with localized stencils and construct the discrete least-squares system for the cardinal basis. This variant demonstrates superior stability and consistent high-order convergence for elliptic PDEs, particularly on domains with Neumann and mixed boundaries where collocation is unstable [2003.03132].
- Partition-of-unity methods construct local RBF–RLS approximants on overlapping patches, blending them with smooth weight functions. Least-squares enforcement at an oversampled set of test points yields error estimates driven by local accuracy and an oversampling-controlled stability norm. This approach is robust against node layout and geometric irregularities, with demonstrated algebraic and spectral convergence in 2D and 3D Poisson problems [1702.07148].

## 5. Applications: Function Approximation, PDEs, and Beyond

RBF–RLS is applied to:
- High-fidelity function approximation on bounded and irregular domains, achieving geometric convergence for smooth targets [2211.12598, 2308.14490].
- Strong-form PDE solution, including elliptic (Poisson, Helmholtz), parabolic, and transport equations, via collocation or weak-form/L$^2$ residual minimization [2211.12598, 1702.07148, 2003.03132].
- Handling of singular sources: The RBF–RLS (single-layer RBF network) is well-suited for PDEs with Dirac delta terms in forcing or boundary/initial data. By direct analytic integration in the weak form, residuals associated with singular sources can be driven to zero, unlike PINNs, whose global neural tangent kernel (NTK) inhibits this decoupling. RBF–RLS maintains high accuracy in forward and inverse problems for advection–dispersion with delta sources [2606.12735].

The approach extends to inverse problems, where model parameters (velocity, diffusivity, reaction rates) are estimated by embedding the RBF–RLS solve within an outer optimization loop, using analytic gradients available from the linear system structure [2606.12735].

## 6. Statistical Foundations and Relationship to Kernel Methods

RBF–RLS and its variant, Least Squares Support Vector Regression (LS-SVR) with RBF kernels, exhibit precise equivalence to Maximum a Posteriori (MAP) inference in Bayesian RBF networks with Gaussian prior on regression weights [1905.00332]:
\[
\min_{w,b,e}\,\frac12\|w\|^2+\frac{\gamma}{2}\sum_{i=1}^N e_i^2 \quad
\longleftrightarrow\quad
\min_{\theta}\,\frac{1}{2\sigma_p^2}\|w\|^2+\frac{1}{2\sigma_n^2}\sum_{i=1}^N(y_i-\phi_i^T\theta)^2,
\]
with the regularization parameter $\gamma$ corresponding to the prior-to-noise variance ratio. The LS-SVR dual is identical to the equation solved by the RBF–RLS method; the predictive mean and variance of the Bayesian model match the LS-SVR predictor and provide explicit posterior uncertainty quantification [1905.00332].

This connection provides theoretical underpinning for the regularization and generalization behavior of RBF–RLS, enables hyperparameter optimization via evidence maximization, and directly relates the design of RBF kernels in deterministic numerical analysis to those in probabilistic inference.

## 7. Numerical Results and Implementation Guidelines

Empirical studies confirm the theoretical predictions:
- Global RBF–RLS systems, with optimal shape parameter scaling and sufficient oversampling, achieve geometric convergence to machine precision with coefficient norms remaining $\mathcal O(1)$ [2211.12598].
- FFT-accelerated RBF–RLS achieves uniform accuracy and moderate solution time for both function approximation and PDEs, surpassing standard dense solvers beyond moderate $N$ [2308.14490].
- Local and PUM RBF–RLS solvers sustain high-order and spectral rates with speedups up to $8\times$ over collocation on the same geometry/patch configuration, and maintain stability under node/h-refinement [1702.07148].
- In singular-source PDEs, RBF–RLS outperforms PINNs by an order of magnitude in RMSE and achieves robust parameter recovery in inverse modeling [2606.12735].
- Key implementation best practices include using an oversampling ratio $\beta\approx 2$ for global methods, placing an extension layer of centers, using QR/SVD regularization at threshold $\tau\sim10^{-10}$, and (for partition/FD methods) careful control of node placement and patch overlap [2211.12598, 1702.07148, 2003.03132].

### Summary Table: Principal RBF–RLS Paradigms

| Variant           | Domain/Structure         | Regularization/Acceleration | Stability Mechanisms            |
|-------------------|-------------------------|-----------------------------|---------------------------------|
| Global RBF–RLS    | Bounded/unstructured    | SVD/QR, oversampling, FFT   | Shape scaling, halo layers      |
| Local RBF–FD–RLS  | Irregular, stencils     | Local QR/SVD, sparse normal | Oversampling, local polynomials |
| RBF–PUM–RLS       | Patches, PUM structure  | Local QR, partitioned solve | Shepard weights, overlap        |

These frameworks collectively underpin a versatile, stable, and theoretically justified suite of mesh-free least-squares solvers for approximation, deterministic PDEs, and probabilistic modeling, with robust performance in both classical and modern scientific computing applications [2211.12598, 2308.14490, 2003.03132, 1702.07148, 2606.12735, 1905.00332].

Source: https://www.emergentmind.com/topics/radial-basis-function-residual-least-squares-rbf-rls