---
title: Neural Basis Functions
url: https://www.emergentmind.com/topics/neural-basis-function
type: topic
---

# Neural Basis Functions

Searching arXiv for recent and foundational papers on neural basis functions, function encoders, radial basis layers, and basis-learning operator methods.
Neural basis function denotes a family of representations in which a model is organized around learned basis elements whose linear or multilinear combinations approximate functions, operators, signals, or control laws. Across recent literature, the term covers several closely related constructions: function encoders that learn basis functions $\{\psi_j\}_{j=1}^n$ spanning a Hilbert-space subspace [2509.20605]; radial-basis-function layers that expand inputs through localized responses before linear recombination [2311.16148]; adaptive basis layers for functional data in which each basis function is itself a micro neural network [2106.10414]; operator-learning architectures that separate basis learning from coefficient mapping [2410.00171, 2212.07336]; and domain-specific basis models for PDEs, neural fields, communications, and control [2601.12704, 2504.13379, 2408.07692, 2506.13168]. The unifying idea is that the expressive nonlinear component is concentrated in the learned basis functions, while prediction, reconstruction, or operator application is carried out in a lower-dimensional coefficient space.

## 1. Conceptual definition and scope

In the most explicit function-space formulation, a function encoder learns basis functions
\[
\{\psi_j\}_{j=1}^n,
\]
which define the feature map
\[
\phi(x)=\big[\psi_1(x),\ldots,\psi_n(x)\big]^\top,
\]
and represent functions in the learned span as
\[
f(x)=\langle c,\phi(x)\rangle
\]
for coefficients $c\in\mathbb{R}^n$ [2509.20605]. With sampled data $\{(x_i,y_i)\}_{i=1}^m$, the coefficients are obtained by regularized least squares,
\[
\min_{c\in\mathbb{R}^n}\ \frac{1}{m}\sum_{i=1}^m\big(y_i-\langle c,\phi(x_i)\rangle\big)^2+\lambda\|c\|^2,
\]
with corresponding normal equations
\[
\left(\frac{1}{m}\sum_{i=1}^m \phi(x_i)\phi(x_i)^\top+\lambda I\right)c = \frac{1}{m}\sum_{i=1}^m y_i\phi(x_i)
\]
[2509.20605]. In this formulation, the neural network is not used as an end-to-end predictor in the usual sense; rather, it learns a basis for a function space, and prediction is a linear solve in that basis [2509.20605].

A broader formulation appears in multivariate approximation. NeuApprox introduces the neural basis function by leveraging an untrained neural network as the basis function, using an MLP-based map
\[
f_\theta : [0,1] \to \mathbb{R}^{N},
\]
typically written as
\[
f_\theta(x) = \boldsymbol{W}_L \big(\cdots \sigma(\boldsymbol{W}_0 x + \boldsymbol{b}_0) \cdots \big) + \boldsymbol{b}_L
\]
with periodic activations such as $\sin(\cdot)$ to better capture high-frequency details [2603.13311]. In that setting, multivariate functions are decomposed into sums of block terms built from products of univariate neural basis functions and coefficient tensors [2603.13311].

Other papers use the same underlying principle with a more localized inductive bias. The U-RBF layer treats each input dimension separately, assigns to each dimension a set of Gaussian RBF neurons, maps each scalar input coordinate into a higher-dimensional representation, and then feeds that expanded representation into a fully connected layer [2311.16148]. NeuRBF writes the representation explicitly as a weighted sum of adaptive radial basis functions centered at learned locations with learned shapes,
\[
g_b(\mathbf{x}) = \sum_{i \in U(\mathbf{x})} \varphi(\mathbf{x}, \mathbf{c}_i, \Sigma_i)\,\mathbf{w}_i,
\]
thereby casting neural fields as basis aggregation followed by decoding [2309.15426].

Taken together, these formulations suggest a common definition: a neural basis function is a learnable basis element, usually parameterized by a neural network or embedded in a neural architecture, whose outputs serve as coordinates, modes, kernels, or localized responses for representing structured families of functions.

## 2. Algebraic and kernel-theoretic formulations

The algebraic form of neural basis-function models is usually a linear or multilinear expansion over learned basis responses. Function encoders use the primal form $f(x)=\langle c,\phi(x)\rangle$ with coefficients computed analytically or by regularized least squares [2509.20605]. Basis-to-Basis (B2B) operator learning represents input and output functions as
\[
f(x) = \sum_{j=1}^{k} \alpha_j\, g_j(x \mid \theta_j), \qquad
(\mathcal Tf)(y) = \sum_{j=1}^{\ell} \beta_j\, h_j(y \mid \vartheta_j),
\]
then learns the map $\alpha\mapsto\beta$ in coefficient space [2410.00171]. BelNet similarly reconstructs operators in a low-rank basis-enhanced form, starting from
\[
\kappa(x, y) = \sum_{k=1}^K p_k(y) q_k(x),
\]
so that
\[
G(u)(x) = \sum_{k=1}^K q_k(x)\int p_k(y)u(y)\,dy
\]
and its neuralized version learns both projection and construction functions directly [2212.07336].

A major development is the kernel interpretation of learned neural bases. Once $\phi$ is learned, function encoders induce the kernel
\[
k(x,x')=\langle \phi(x),\phi(x')\rangle=\sum_{j=1}^n \psi_j(x)\psi_j(x'),
\]
which is automatically symmetric positive semidefinite and therefore a valid reproducing kernel [2509.20605]. In the dual, the same predictor can be written as
\[
w^*=\sum_{j=1}^m \alpha_j \phi(x_j),
\]
with coefficients satisfying
\[
(K+\lambda m I)\alpha = Y,\qquad K_{ij}=k(x_i,x_j)
\]
[2509.20605]. The vector-valued extension replaces the scalar kernel by the operator-valued kernel
\[
\kappa(x,x')=\sum_{j=1}^n \psi_j(x)\psi_j(x')^\top
\]
[2509.20605].

This kernel-theoretic reading is important because it connects learned neural bases to classical RKHS reasoning while preserving the computational advantage of a compact primal representation. In the function-encoder setting, test-time cost is naturally $\mathcal{O}(n)$ per test point rather than scaling with the number of training examples $m$ [2509.20605]. A plausible implication is that neural basis-function models occupy an intermediate position between fixed-basis approximation and data-dependent kernel learning: the basis is explicit and finite-dimensional, but the induced similarity structure is learned rather than prescribed.

## 3. Architectural families

The literature contains several distinct architectural realizations of neural basis functions.

Function encoders and related Hilbert-space models learn neural network basis functions and then solve for coefficients by least squares or ridge regression [2509.20605]. The central object is the learned span itself, not an end-to-end predictor. B2B operator learning extends this idea to paired input and output spaces, with separate learned bases and a coefficient-space operator, and derives algorithms directly analogous to eigendecomposition and singular value decomposition for linear operators [2410.00171].

Functional-data models realize the basis directly inside a neural network layer. AdaFNN introduces a Basis Layer whose hidden units are each basis functions themselves implemented as a micro neural network. For a functional input $X(t)$, basis coefficients are computed as
\[
c_i = \langle \beta_i, X\rangle = \int \beta_i(t)\,X(t)\,dt,
\]
where each $\beta_i(t)$ is parameterized as
\[
\beta_i(t) = nn_{\Theta_i}(t)
\]
[2106.10414]. The low-dimensional coefficient vector is then passed to a standard feedforward network [2106.10414].

Radial-basis architectures use localized functions as the basis family. The U-RBF layer maps each scalar input coordinate into Gaussian responses
\[
z_{d,k} = \exp\left(-\frac{(x_d - c_{d,k})^2}{2o_{d,k}^2}\right),
\]
then forms the next-layer output by weighted summation over these activations [2311.16148]. Classical and adaptive RBF networks appear in several other forms: Beta Basis Function Neural Networks use beta transfer functions in the hidden layer and linear output combinations [1810.13135]; DeepLABNet replaces fixed activations with feature-wise scalar RBF interpolators of the form
\[
y = f(x) = \sum_{i=1}^{s} \lambda_i \,\phi(|x - c_i|) + v_0 x + v_1
\]
[1911.09257]; NeuRBF uses adaptive centers $\mathbf{c}_i$ and anisotropic shapes $\Sigma_i$ with inverse-quadratic kernels
\[
\varphi(\mathbf{x}, \mathbf{c}_i, \Sigma_i) = \frac{1}{1 + (\mathbf{x}-\mathbf{c}_i)^T \Sigma_i^{-1} (\mathbf{x}-\mathbf{c}_i)}
\]
and optional normalized weights [2309.15426].

Complex-valued basis-function networks form another family. Phase-transmittance radial basis function neural networks use complex-valued inputs, outputs, weights, biases, center vectors, and variances, with split-complex radial kernels that propagate real and imaginary parts separately [2408.07692]. Deep complex-valued RBF networks extend the classical shallow C-RBF by stacking multiple Gaussian basis layers
\[
\mathbf{y}^{\{l\} = \mathbf{W}^{\{l\}\boldsymbol{\phi}^{\{l\} + \mathbf{b}^{\{l\},
\qquad
\phi_m^{\{l\} = \exp\!\left[-\frac{\|\mathbf{y}^{\{l-1\}-\boldsymbol{\gamma}_m^{\{l\}\|_2^2}{\sigma_m^{\{l\}}\right]
\]
[2408.16778].

A useful summary is that neural basis-function architectures differ chiefly in what is learned as the basis element: global Hilbert-space modes [2509.20605], inner-product basis functions for functional data [2106.10414], localized radial responses [2311.16148, 2309.15426], learned activation functions [1911.09257], operator-domain bases [2410.00171, 2212.07336], or reusable basis modules for initialization and transfer [2510.08780].

## 4. Basis learning, coefficient estimation, and model compression

A recurring feature of neural basis-function methods is the separation between basis learning and coefficient estimation. In B2B, sampled data $\{(x_i,f(x_i))\}_{i=1}^m$ are projected onto the learned basis by solving
\[
\min_{\alpha} \frac{1}{m} \sum_{i=1}^{m} \left\| f(x_i) - \sum_{j=1}^{k}\alpha_j g_j(x_i \mid \theta_j)\right\|_2^2,
\]
with closed-form solution
\[
\alpha = (G^\top G)^{-1}G^\top \boldsymbol f
\]
[2410.00171]. Function encoders use the analogous regularized least-squares solve in the learned feature space [2509.20605]. BelNet computes latent coefficients from arbitrary input samples and reconstructs output functions through learned basis functions on the output domain [2212.07336]. In the reusable initialization framework, pretrained neural basis networks approximate polynomial basis functions on a reference domain and are then assembled for target functions as
\[
\mathcal{P}f(x) = \sum_{k=0}^{K} \alpha_k \hat{\varphi}_k(\mathcal{T}^{-1}(x)),
\]
with coefficients usually computed by least squares [2510.08780].

Compactness and intrinsic dimension are major concerns. Function encoders propose two PCA-guided schemes for learning compact bases [2509.20605]. In progressive training, basis functions are learned sequentially, coefficients are computed across training tasks, and the covariance
\[
\Sigma_b=\frac{1}{N-1}\sum_{i=1}^N (c^i-\bar c)(c^i-\bar c)^\top
\]
is used to compute explained variance ratios $\mathrm{EVR}_k=\lambda_k/\sum_i\lambda_i$; training stops when cumulative explained variance exceeds a threshold $\tau$ such as $99\%$ [2509.20605]. In train-then-prune, an overparameterized basis set of size $B$ is trained jointly, PCA is applied to the learned coefficient covariance $\Sigma_B$, the effective rank is inferred as
\[
r=\min\left\{n:\frac{\sum_{i=1}^n \lambda_i}{\sum_{j=1}^B \lambda_j}\ge \tau\right\},
\]
and the bases are scored by
\[
s_p=\sum_{i=1}^r \lambda_i\,U_{pi}^2
\]
before pruning and fine-tuning [2509.20605].

The polynomial benchmark in the same paper provides a particularly clean example of intrinsic-dimension recovery. Degree-$d$ polynomials have intrinsic dimension $d+1$, and the eigenvalue scree plots show elbows exactly at that dimension; the progressive method stops after $4$, $5$, and $6$ bases for degrees $3$, $4$, and $5$, respectively, while train-then-prune identifies the same cutoffs [2509.20605].

This use of PCA is not merely a visualization device. It functions as a basis-selection criterion revealing how many learned modes are actually required to span the dominant variability of a function family. This suggests a broader interpretation of neural basis-function learning as supervised or task-aware dimension reduction in function space.

## 5. Theoretical properties

Several papers place neural basis functions on a formal approximation-theoretic footing. NeuApprox states a Neural Approximation Theorem: for any continuous multivariate function $\psi:[0,1]^n\to\mathbb{R}$ and any desired accuracy $\epsilon>0$, there exists a finite set of neural basis functions and corresponding coefficient tensors such that
\[
\sup_{\boldsymbol{x}\in[0,1]^n}\left|\psi(\boldsymbol{x})-f_\Theta(\boldsymbol{x})\right|<\epsilon
\]
for
\[
f_{\Theta}(\boldsymbol{x})=\sum_{j=1}^{T}\mathcal{C}^j\times_1 f_{\theta_1^j}(x_1)\times_2 f_{\theta_2^j}(x_2)\cdots\times_n f_{\theta_n^j}(x_n)
\]
[2603.13311]. The proof combines Stone–Weierstrass density of separable products with universal approximation for the univariate neural factors [2603.13311].

AdaFNN provides a consistency result for functional mappings $T = h\circ g$, where $g:C([0,1])\to \mathbb{R}^q$ is a finite-dimensional linear continuous map and $h:\mathbb{R}^q\to \mathbb{R}$ is continuous nonlinear. Under accurate numerical integration and sufficient network capacity, there exists a network $\widehat{T}^\ast$ such that
\[
\sup_{f\in C([0,1]),\ \|f\|_2\le 1} \left|\widehat{T}^\ast(f)-T(f)\right|<\epsilon
\]
for any $\epsilon>0$ [2106.10414]. The same paper also derives a small-generalization-error result, with expected generalization gap bounded as
\[
\left| \mathbb{E}_{S,A}\big[r(\widehat{\Theta}) - r_n(\widehat{\Theta})\big]\right| \lesssim \frac{T^{\,1-1/c}}{n}
\]
under compactness and Lipschitz assumptions with SGD step sizes $\alpha_t\le c/t$ [2106.10414].

Function encoders develop finite-sample guarantees for the fixed-basis stage. If each basis is bounded by $\sup_x |\psi_j(x)|\le R$ and outputs satisfy $\|y\|\le Y$, the learned predictor obeys a Rademacher-style risk bound of the form
\[
L(f_{\hat c_\lambda}) \le \hat L_m(f_{\hat c_\lambda}) + 2Y^2R\sqrt{\frac{n}{m\lambda}\Bigl(R\sqrt{\frac{n}{\lambda}+1\Bigr)\Bigl(2+\sqrt{\frac{\log(1/\delta)}{2}\Bigr)}
\]
and asymptotically
\[
L(f_{\hat c_\lambda}) \lesssim \hat L_m(f_{\hat c_\lambda}) +\tilde{\mathcal O}\!\left(\frac{Y^2R^2\,n}{\lambda\sqrt m}\right)
\]
[2509.20605]. The same analysis yields
\[
\mathcal R_m(\mathcal F_{\mathcal C_\lambda}) \in \mathcal O\!\left(\sqrt{\frac{n}{m\lambda}}\right),
\]
making explicit the tradeoff between basis count, data size, and regularization [2509.20605]. A PAC-Bayes bound is also derived using truncated Gaussian priors and posteriors over coefficients, with a key KL estimate $D_{KL}(\nu\|\nu_0)\lesssim 2n$ [2509.20605].

Operator-learning theory supplies another layer of structure. For linear operators, B2B proves that if $f_3 = af_1 + bf_2$ and $\mathcal T$ is linear, then the learned operator satisfies
\[
\hat{\mathcal T}f_3 = a\,\hat{\mathcal T}f_1 + b\,\hat{\mathcal T}f_2
\]
because the coefficient map is linear and least-squares projection is linear in the data [2410.00171]. The same framework gives explicit SVD-style and eigendecomposition-style operator approximations in learned bases [2410.00171].

These results do not imply identifiability of individual learned basis functions. AdaFNN states explicitly that the consistency theorem is for the network map $\widehat{T}$ and does not mean the individual basis functions are identifiable, since many parameterizations can represent the same map [2106.10414]. That point addresses a common misconception: neural basis functions are often meaningful as spans or coordinate systems even when particular basis elements are non-unique.

## 6. Applications and empirical behavior

Neural basis-function methods have been applied to regression, functional data analysis, operator learning, PDE solution, control, communications, and scientific computing.

In low-dimensional regression, U-RBF is evaluated on low-dimensional function regression, image regression, and PMLB datasets with at most $5$ input dimensions. It is often the best overall among deep learning methods on the real-world low-dimensional regressions, performs very well on isotropic low-pass filtered white noise, and is competitive but not always superior on coordinate-to-RGB image regression, where Fourier-feature methods often perform best [2311.16148]. The same paper emphasizes that no dedicated reinforcement-learning experiment is reported, despite control being mentioned as a general motivation [2311.16148].

In function-space representation, function encoders validate compact-basis learning on polynomial spaces and nonlinear dynamical systems. On the Van der Pol oscillator, prior work used $100$ neural ODE bases, while the PCA-guided methods find that only $2$ bases are sufficient to capture nearly all variance and attain comparable predictive accuracy; for the planar two-body problem, about $5$–$6$ bases capture over $99\%$ of the variance and reproduce the orbital dynamics accurately [2509.20605].

In operator learning, B2B reports a two-orders-of-magnitude improvement in accuracy over existing approaches on several benchmark tasks and is especially strong on linear operators such as derivative and anti-derivative, while remaining robust to variable sensor locations that prevent DeepONet variants from converging [2410.00171]. BelNet targets mesh-free operator learning for parametric PDEs and reports lower mean relative error than DON on several multiscale and high-contrast problems, including multiscale elliptic equations and Richard’s equation, with the free-BelNet variant benefiting from varying input sensors [2212.07336].

In functional data analysis, AdaFNN compares learned basis layers against raw discretized inputs, B-spline bases, and FPCA-based pipelines. It is reported as the best method on all nine real-data prediction tasks in the reported table, and it often achieves better accuracy with fewer basis functions than B-spline or FPCA pipelines [2106.10414]. Orthogonality regularization often improves performance, while sparsity regularization mainly helps interpretability and sometimes predictive accuracy [2106.10414].

In neural fields and coordinate-based representations, NeuRBF demonstrates higher accuracy and compactness than prior methods on 2D image fitting, 3D signed distance fields, and neural radiance field reconstruction. Reported examples include about $58.56$ dB PSNR on DIV2K, average IoU $0.9995$ and normal angular error $4.93$ on 3D SDF tasks, and average PSNR $34.62$, SSIM $0.975$, and LPIPS$_{VGG}$ $0.034$ on synthetic NeRF scenes [2309.15426].

In scientific computing, reusable neural basis pretraining on polynomial families yields strong 1D and 2D approximation results and improved extrapolation through a domain mapping mechanism [2510.08780]. RRNN solves multiscale elliptic equations by randomized Gaussian RBF bases on domain-decomposed subproblems, reducing training to a least-squares solve for output-layer weights [2407.14745]. PIRBFNN solves Black–Scholes PDEs for single-asset, double-asset, and four-asset options by jointly optimizing RBF centers, shape parameters, weights, and network width under a physics-informed loss, with residual-based adaptive neuron insertion [2601.12704]. For neural field equations on curved cortical surfaces, local RBF interpolatory projection combined with RBF quadrature provides a mesh-flexible, high-order framework capable of simulating labyrinthine patterns and traveling spots on deformed spheres, tori, and realistic cortical meshes [2504.13379].

In control and communications, TGRBF combines an RBFNN branch with a GRU branch through a dynamic gate,
\[
y_{\text{tgrbf}(t)}=g(t)\cdot y_{\text{rbf}(t)} + (1-g(t))\cdot y_{\text{gru}(t)},
\]
and is used for system identification and Jacobian-driven adaptive control with Lyapunov-based guarantees of uniform ultimate boundedness [2506.13168]. In 5G-style communication tasks, deep PT-RBF and deep C-RBF networks require carefully derived initialization schemes; random and constellation-based initialization fail to converge in deep settings, whereas the proposed statistically informed initialization is reported as the only one to achieve successful convergence for the tested deep architectures [2408.07692, 2408.16778].

## 7. Related concepts, misconceptions, and open directions

A common misconception is that neural basis functions are merely classical RBF networks under a new name. The literature shows a broader picture. Some models do use Gaussian, inverse-quadratic, or beta basis functions [2311.16148, 2309.15426, 1810.13135], but others learn basis functions as general neural networks in Hilbert spaces [2509.20605], as micro networks inside a Basis Layer [2106.10414], or as reusable modules pretrained on polynomial families [2510.08780]. The term therefore encompasses both localized radial bases and nonlocal learned modes.

A second misconception is that the basis must be fixed once learned. Several papers emphasize adaptability after pretraining. The reusable initialization framework transfers pretrained polynomial basis networks to new domains using a domain transform and least-squares coefficient fitting [2510.08780]. NeuApprox compares freezing, fine-tuning with LoRA (rank $10$), and training from scratch, reporting that fine-tuning provides a better speed/accuracy trade-off than training from scratch on out-of-distribution data [2603.13311]. TGRBF performs event-triggered online optimization with an experience buffer and momentum-explicit gradient descent to refine parameters during control execution [2506.13168].

There are also objective limitations. U-RBF imposes an axial or univariate bias because each dimension is processed separately, which is helpful only when the problem aligns with that structure [2311.16148]. DeepLABNet reports slower training, increased overfitting with model size, higher performance variance than ReLU, and greater sensitivity to learning-rate scheduling [1911.09257]. Function encoders make the expressivity–complexity tradeoff explicit through the dependence of generalization bounds on basis count $n$ [2509.20605]. Deep complex-valued RBF models and PT-RBFs highlight that poor initialization can prevent convergence entirely in multilayer settings [2408.16778, 2408.07692].

A plausible synthesis is that neural basis functions are best understood not as a single architecture but as a design principle: learn a compact, adaptive set of nonlinear basis elements whose coefficients can be solved, regressed, optimized, or controlled in a lower-dimensional space. Recent work pushes this principle in three directions. One is toward functional analysis and statistical guarantees, especially through function encoders and B2B [2509.20605, 2410.00171]. Another is toward task-structured and geometry-aware bases for scientific computing, operator learning, and neural fields [2212.07336, 2601.12704, 2504.13379]. A third is toward modularity, transfer, and fine-tuning, where pretrained basis libraries become reusable building blocks rather than task-specific endpoints [2510.08780, 2603.13311].

Source: https://www.emergentmind.com/topics/neural-basis-function