Neural Approximation (NeuApprox)

• Neural Approximation (NeuApprox) is a research framework that represents functions, operators, and decision rules with neural networks while balancing expressivity, stability, and efficiency.
• It spans approximation theory, neural basis function paradigms, and operator learning, with applications in image completion, PDE solutions, and adaptive control.
• The approach leverages universal approximation results and quantitative error rates to guide architecture design and achieve practical performance in multivariate data tasks.

Neural Approximation (NeuApprox) denotes a family of closely related research directions concerned with representing functions, operators, and decision rules by neural networks, and with quantifying when such representations exist, how the approximation error scales with architectural complexity, and how these approximants can be deployed in practice. In current usage, the term has been applied both to a broad approximation-theoretic program—covering universal approximation, approximation rates, approximation spaces, and architectural expressivity—and to a specific neural basis function framework for multivariate data representation (Nishijima, 2021, Gribonval et al., 2019, Wu et al., 4 Mar 2026).

1. Terminological scope and conceptual range

The literature does not treat NeuApprox as a single standardized architecture. Instead, the term spans several levels of abstraction, from general function-space theory to concrete algorithmic systems. Taken together, these works suggest that NeuApprox is best understood as a research program centered on neural representations of mathematical objects and on the trade-offs between expressivity, stability, trainability, and computational efficiency.

Usage	Representative object	Representative sources
Approximation theory of neural networks	Density in $C(K)$ and $L^p$, rates on Sobolev/Barron classes, approximation spaces	(Nishijima, 2021, DeVore et al., 2020, Gribonval et al., 2019, Mukherjee et al., 20 May 2026)
Neural basis-function paradigm	Multivariate function decomposition into block terms with univariate neural bases and coefficient tensors	(Wu et al., 4 Mar 2026)
Task-specific neural approximation systems	Approximate computing, operator learning, control, finance, and numerical discretization	(Peng et al., 2018, Huang et al., 30 Nov 2025, Furuya et al., 2024, Stannat et al., 2023)

This range matters because different papers optimize different objects. Some study approximation in $\sup$-norm or $L^p$; some define approximation spaces through best-approximation decay; some approximate operators between function spaces; some approximate optimal feedback laws or superhedging prices; and some replace hand-crafted bases or virtual-element basis functions with learned neural surrogates. The AXNet paper is particularly explicit that it does not reference or compare to a method termed “NeuApprox,” even though it belongs to the broader class of neural approximation frameworks (Peng et al., 2018).

2. Universal approximation and function-space formulations

A canonical starting point is the one-hidden-layer model

$F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$

For compact $K \subset \mathbb{R}^r$, the classical density statement is that such networks are dense in $C(K)$ under appropriate conditions on $\sigma$. In Nishijima’s formal treatment, continuous sigmoidal activations are discriminatory and therefore yield density in $C(K)$ via Hahn–Banach separation and the Riesz–Markov–Kakutani representation; more generally, Leshno et al.’s condition is that $\sigma$ must not coincide almost everywhere with a polynomial (Nishijima, 2021). The same source develops extensions to vector-valued targets, deeper feedforward compositions, and $L^p$0 approximation under finite-measure assumptions.

The proof strategies vary but are structurally consistent. Stone–Weierstrass arguments establish density when ridge-function algebras separate points and contain constants; measure-separation arguments show that failure of density would imply a nonzero functional annihilating the network span; finite-difference constructions recover monomials from translated activations; and convolution-based smoothing extends results from smooth activations to broader measurable classes (Nishijima, 2021). These are representational statements: they specify when approximation exists, not how the parameters are found.

Quantitative versions refine this existence theory. One survey states that, for polynomial targets of total degree at most $L^p$1 in $L^p$2 variables, a single-hidden-layer network with $L^p$3 hidden units suffices, independently of output dimension; for general continuous targets on compact $L^p$4, one can take $L^p$5 hidden units to achieve uniform error $L^p$6 (Chong, 2020). A complementary survey records that deep narrow networks with Leaky-ReLU are dense in $L^p$7 if and only if the width satisfies

$L^p$8

and, in particular, scalar-output universality is achieved at width $L^p$9 (Mukherjee et al., 20 May 2026).

An alternative formulation weakens the metric rather than the model. Thom studies measurable functions on $\sup$0 and defines the observer-aware metric

$\sup$1

With $\sup$2, every measurable $\sup$3 admits a $\sup$4-representable $\sup$5 satisfying $\sup$6, yielding a dimension-independent regularity-style decomposition into a structured part and a residual invisible to bounded-complexity neural observers (Thom, 2019).

3. Quantitative rates, approximation spaces, and architectural expressivity

Universality says little about efficiency. The rate question asks how the best approximation error decays as width, depth, or parameter count increase. On Sobolev classes $\sup$7, Nishijima reports the Jackson-type bound

$\sup$8

together with the lower bound

$\sup$9

which exhibits the classical curse of dimensionality (Nishijima, 2021). By contrast, for Barron-type classes defined through the $L^p$0-variation seminorm,

$L^p$1

the error in Hilbert norms scales like $L^p$2, independent of ambient dimension, and $L^p$3 variants yield $L^p$4 rates (Nishijima, 2021). A recurring misconception is thereby ruled out: dimension-free neural approximation is not generic; it is tied to special function classes such as Barron spaces.

This rate viewpoint is formalized further through approximation spaces. For a family $L^p$5, the approximation quasi-norm is

$L^p$6

Approximation spaces of deep neural networks then appear as $L^p$7 and $L^p$8, depending on whether complexity is measured by connections or neurons (Gribonval et al., 2019). This framework yields several structural facts: generalized networks with identity channels have the same approximation spaces as strict networks; bounded depth classes depend only on the asymptotic depth-growth equivalence class; and, for ReLU powers, polynomial depth growth leads to saturation, so $L^p$9 and higher-degree truncated powers generate the same approximation spaces (Gribonval et al., 2019).

ReLU geometry explains why depth alters these spaces. A fixed-width, fixed-depth ReLU network realizes a continuous piecewise linear function subordinate to a convex-polytopal partition. The number of linear regions can grow exponentially with depth, and the resulting realization set forms a parametric nonlinear manifold with pronounced space-filling behavior (DeVore et al., 2020). That geometry improves rate–distortion on some classes, but it also creates numerical instability: stable manifold widths tie achievable approximation rates to entropy numbers, and local Lipschitz continuity of the realization map does not translate into globally stable parameter selection (DeVore et al., 2020).

Two recent constructive strands sharpen the depth story. Daubechies and coauthors show that refinable functions, including wavelet scaling functions and subdivision limits, are approximated by fixed-width deep ReLU networks with exponentially decaying error in the number of parameters, via a construction that emulates the cascade algorithm (Daubechies et al., 2021). Independently, a wavelet-frame approach builds single-hidden-layer W⃗B-Nets whose approximation error on the frame-sparse class $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$0 satisfies

$F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$1

and extends this guarantee to non-smooth activations through the explicit $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$2 distance between a smooth activation $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$3 and a surrogate $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$4 (Hur et al., 23 Apr 2025). This suggests that constructive neural approximation can be organized not only by generic width/depth counts, but also by analytic bases and multiscale structure.

4. NeuApprox as a neural basis-function decomposition framework

A narrower and explicitly named usage appears in the work “Neural Approximation and Its Applications,” where NeuApprox is introduced as a neural basis function-driven paradigm for multivariate function approximation and multi-dimensional data representation (Wu et al., 4 Mar 2026). The central object is a block-term decomposition

$F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$5

in which $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$6 is a learnable coefficient tensor and each $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$7 is a univariate neural basis function, typically an MLP with periodic activation. When $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$8 for all $F_m(x)=\sum_{j=1}^m c_j \sigma(w_j^\top x+b_j).$9, the model reduces to CP decomposition; when $K \subset \mathbb{R}^r$0, it reduces to Tucker form.

This formulation is motivated by limitations of hand-crafted bases such as polynomial, Fourier, Chebyshev, Hermite, and wavelet systems. NeuApprox replaces those fixed dictionaries with untrained, then optimized or fine-tuned, neural basis functions. The theoretical support is twofold. First, univariate neural basis functions satisfy a universal approximation theorem whenever the activation is non-affine, continuous, and differentiable at at least one point with nonzero derivative. Second, Stone–Weierstrass is invoked on the algebra of finite sums of separable products to prove that the multivariate NeuApprox model can approximate any continuous $K \subset \mathbb{R}^r$1 arbitrarily well (Wu et al., 4 Mar 2026).

Training is problem-specific but structurally uniform. For meshgrid completion tasks, the model minimizes a projection-consistent MSE over observed entries; for off-meshgrid point cloud completion, it minimizes squared reconstruction error over observed coordinates. The paper uses Adam with learning rate $K \subset \mathbb{R}^r$2, weight decay selected from $K \subset \mathbb{R}^r$3, network depth $K \subset \mathbb{R}^r$4, hidden width in $K \subset \mathbb{R}^r$5, and number of block terms $K \subset \mathbb{R}^r$6. For rapid out-of-distribution adaptation, it applies LoRA with rank $K \subset \mathbb{R}^r$7 while always re-optimizing the coefficient tensors (Wu et al., 4 Mar 2026).

Empirically, the framework is evaluated on multispectral images, videos, light fields, traffic tensors, and point clouds. Representative results include PSNR $K \subset \mathbb{R}^r$8, SSIM $K \subset \mathbb{R}^r$9, and NRMSE $C(K)$0 on MSI Toys at sampling rate $C(K)$1; PSNR $C(K)$2, SSIM $C(K)$3, and NRMSE $C(K)$4 on the Origami light field at sampling rate $C(K)$5; RMSE $C(K)$6 and MAPE $C(K)$7 on Guangzhou traffic completion at missing rate $C(K)$8; and average point-cloud NRMSE $C(K)$9 with $\sigma$0 across five datasets (Wu et al., 4 Mar 2026). The same study reports that multiple block terms consistently outperform single-term variants, that neural basis functions outperform polynomial, Fourier, and Gaussian bases, and that fine-tuning pretrained bases can achieve near-scratch performance with markedly reduced time.

5. Operator, dynamical-system, and neural-ODE variants

A substantial branch of NeuApprox concerns operators rather than static functions. One example is the neural oscillator architecture for continuous-time operator learning. Here an input $\sigma$1 is encoded by a second-order ODE

$\sigma$2

and decoded by an MLP

$\sigma$3

Two complementary results are established: an operator-learning bound for causal uniformly continuous maps, and a system-learning bound for uniformly asymptotically incrementally stable second-order systems (Huang et al., 30 Nov 2025). In the first case, the error contains an encoder term decaying like $\sigma$4 and a readout term controlled by the readout depth $\sigma$5; in the second, the widths of the encoder and readout scale like $\sigma$6 and $\sigma$7, leading to polynomial error decay in the reciprocals of the MLP widths. Numerical experiments recover exponents close to $\sigma$8 for the operator-learning depth law and $\sigma$9 for the stable-system width law (Huang et al., 30 Nov 2025).

A related but distinct development gives quantitative approximation rates for solution operators of nonlinear parabolic PDEs by aligning neural operators with Picard iteration in the Duhamel formulation (Furuya et al., 2024). The solution operator is decomposed into a finite-rank approximation of the Green kernel and a one-dimensional neural approximation of the scalar nonlinearity. The resulting neural operator has provable complexity bounds

$C(K)$0

together with a rank choice ensuring the kernel truncation errors $C(K)$1 and $C(K)$2 are at most $C(K)$3 (Furuya et al., 2024). The argument is explicitly dimension-aware but avoids exponential complexity growth by exploiting PDE structure rather than approximating arbitrary operators directly.

A third line embeds approximation theory into the activation itself. In “Approximation properties of neural ODEs,” the activation $C(K)$4 is the time-$C(K)$5 flow of an autonomous neural ODE,

$C(K)$6

and the shallow network is $C(K)$7 (Marinis et al., 19 Mar 2025). The class has the UAP in $C(K)$8 because the scalar flow is nonpolynomial, but the paper also studies stability by constraining the flow Lipschitz constant through the logarithmic-norm quantity $C(K)$9. UAP persists when only the flow Lipschitz constant is constrained, and also when only the linear layers are forced to have unit spectral norm; under simultaneous enforcement, the paper provides upper and lower approximation bounds rather than a UAP theorem. On MNIST, a trained model with $\sigma$0 is post-hoc stabilized to $\sigma$1, and re-training the linear layers yields accuracies $\sigma$2, $\sigma$3, and $\sigma$4, while adversarial robustness under FGSM at $\sigma$5 improves from $\sigma$6 in the original model to $\sigma$7 at $\sigma$8 (Marinis et al., 19 Mar 2025).

6. Scientific-computing and domain-specific realizations

In numerical analysis, NeuApprox appears both as a solver component and as a discretization principle. One thesis develops a Galerkin pipeline for elliptic PDEs by constructing ReLU networks that approximate scalar multiplication, matrix multiplication, and inversion of SPD matrices, and then derives a Besov-to-neural embedding

$\sigma$9

for $L^p$00 (Romera, 2024). The same work gives explicit network sizes for the matrix inverse approximator and translates operator-norm error directly into Galerkin solution error. A complementary polygonal discretization, the Neural Approximated Virtual Element Method, replaces virtual basis functions by learned harmonic-polynomial surrogates and empirically recovers the expected $L^p$01 $L^p$02 and $L^p$03 $L^p$04 rates on quadrilateral meshes, while eliminating explicit VEM projection and stabilization operators (Berrone et al., 2023).

Control theory provides another important realization. For stochastic reaction–diffusion equations with additive noise, optimal controls are reduced to feedback form and then approximated by finitely based neural parameterizations; an adjoint-based Monte Carlo estimator trains the feedback, and the suboptimality bound separates basis truncation error $L^p$05 from neural approximation error $L^p$06 (Stannat et al., 2023). In adaptive neural control, “desired approximation” replaces state-based approximation by approximating $L^p$07 and $L^p$08 using only the desired trajectory as network input. This removes the compact-set prerequisite on the unknown actual state, and the resulting integral and incremental adaptation laws guarantee boundedness and convergence of the filtered tracking error to an adjustable set [(Hur et al., 23 Apr 2025)?]