Mesh-Free Neural Function Approximators

• Mesh-free neural function approximators are advanced neural architectures that learn continuous functions or operators from scattered data without requiring traditional grid-based discretization.
• They employ diverse methods such as Fourier neural networks, RVFL networks, and Fredholm-nets that use spectral, randomization, and integral approaches for universal approximation.
• These techniques offer flexibility for high-dimensional and non-metric inputs while addressing challenges in scalability, high-frequency accuracy, and complex boundary conditions.

Mesh-free neural function approximators are neural architectures and training methodologies that enable function interpolation, regression, or operator learning on continuous or high-dimensional domains without requiring an explicit mesh or grid discretization of the input domain. These frameworks allow the direct use of scattered data, pointwise sampling, or even non-metric or infinite-dimensional inputs, circumventing the need for mesh-generation, domain partitioning, or quadrature integration typically required by classical numerical or finite-element-based function approximation methods.

1. Foundational Principles and Mathematical Formalism

Mesh-free neural function approximators rely on the principle that neural networks with appropriate architecture and training protocols can universally approximate continuous functions, functionals, or solution operators directly from pointwise samples or coordinate features. The mesh-free property is realized by constructing the function representation, loss function, and sampling regime to avoid any dependency on a domain mesh, grid, or triangulation.

Canonical examples include:

• Fourier Neural Networks (FNNs): Single- or multilayer architectures with sinusoidal activations; the FNN’s weights, after training, correspond to frequencies, amplitudes, and phases that mirror the truncated Fourier expansion of periodic functions. Losses are constructed to enforce both data fidelity and periodicity, with inputs sampled randomly—not on a mesh—across the domain (Ngom et al., 2020, Bose et al., 2021).
• Random Vector Functional Link (RVFL) Networks: Shallow architectures with random input-to-hidden weights and biases, and closed-form output-layer solutions, trained on randomly sampled data in the input (and, if applicable, chart) space to approximate on Euclidean domains or embedded submanifolds (Needell et al., 2020).
• Fredholm-net Frameworks: Function approximation reformulated as (mesh-free) regularized Fredholm integral equations in parameter (neuron) space, discretized by (possibly randomized or low-rank) bases and solved using convex optimization and tensor-train techniques (Gelß et al., 2023).
• Infinite-dimensional / Non-metric Neural Architectures: Input and output spaces specified only by separating function sequences; neural mappings constructed via Hilbert-space representations with no mesh, universal approximation theorems, and finite-dimensional projections for practical deployment (Galimberti, 2024).
• Neural Operator Architectures (e.g., BelNet): Operator learners that use mesh-free sensor locations (possibly variable per sample), with learned projections and reconstructions, providing domain- and discretization-independence in operator learning (input-output functions on possibly different domains) (Zhang et al., 2022).

These constructions sidestep mesh- or chart-based decomposition: the function evaluations are parametrized either by feature extractions, sample coordinates, or manifolds’ intrinsic charts, but never via explicit partitioning or meshing of the spatial domain.

2. Representative Architectures and Methodologies

Mesh-free neural approximators can be classified according to architectural and algorithmic strategies:

Approach	Core Feature	Universal Approximation
FNN/Trigonometric Nets	Sinusoidal/cosine activations	Yes: $O(1/N)$ to $O(N^{-p})$ for analytic targets
RVFL	Randomized input-to-hidden	Yes: $O(1/\sqrt{N})$ mean-squared error
Fredholm-nets	Integral operator form	Yes: via Ritz-Galerkin & Tikhonov regularization
BelNet	Latent-projection + learned basis	Yes: operator UAP extension (Chen & Chen)
Dimension-free Deep Nets	Affine+clipped activation, $\infty$-width	Yes: width/depth depends only on error, not on $n$
Infinite-dimension Hilbert	Feature injection into $\ell^2$	Yes: for quasi-Polish/Fréchet spaces
XNet/Cauchy Kernel Nets	Cauchy activations, single-layer	Yes: arbitrary-order polynomial rate, $O(N^{-p})$

Training and sampling procedures are fundamentally mesh-free: collocation points or sensors are drawn independently in the domain (or manifold chart), and network parameters are fit via SGD, convex solvers, or analytic output-layer regression, with no dependency on regular grid layouts or triangulations.

3. Theoretical Guarantees and Universality

Universal approximation theorems for mesh-free architectures are grounded in one of several mechanisms:

• Spectral-based UAP: FNNs, RVFLs, Fredholm-nets, BelNet, and XNet each provide universal approximation on compact domains, with explicit bounds for smoothness classes, often dimension-independent for particular spectral-norm settings (Ngom et al., 2020, Bose et al., 2021, Needell et al., 2020, Gelß et al., 2023, Zhang et al., 2022, Yang et al., 2022, Li et al., 31 Jan 2025, Thom, 2019).
• Infinite-dimensional Inputs/Outputs: The Galimberti architecture proves UAP for quasi-Polish inputs and outputs, generalizing to Fréchet/Banach spaces, with stable finite-dimensional realization and numerical feasibility (Galimberti, 2024).
• Functionals and Operator Learning: Operator UAPs extend classical UAP to maps between spaces of functions, with Barron-spectral and Cauchy-integral-based constructions showing $O(1/\sqrt{m})$ or $O(m^{-p})$ convergence rates independent of ambient/intrinsic dimension (Zhang et al., 2022, Yang et al., 2022, Li et al., 31 Jan 2025).

A defining feature is the decoupling of approximation error from domain mesh-size or discretization. In several results, error and parameter count depend only on function class or desired accuracy, not on input dimension or grid resolution (Thom, 2019, Yang et al., 2022, Galimberti, 2024).

4. Mesh-Free PDE Solving and High-Dimensional Surrogate Modeling

Mesh-free neural function approximators have been deployed as direct PDE solvers, surrogate models, and operator learners for high-dimensional scientific computing:

• PINN-style Collocation: Losses are constructed as L²-, $H^1$-, or residual-norms over randomly sampled points, allowing direct enforcement of PDE operators, boundary/interface conditions, and jump conditions without mesh-based quadrature (Ngom et al., 2020, He et al., 2020, Bose et al., 2021, Zhang et al., 2022, Joglekar et al., 2023).
• Piecewise DNNs for Interfaces: Networks are independently parameterized in sub-domains, with adaptive mesh-free sampling near interfaces (monitored by local residuals), achieving convergence rates competitive with mesh-based and XFEM approaches but with more flexibility for complex geometries (He et al., 2020).
• Operator Learning (BelNet): Direct learning of operators $G:V \to U$ where $V,U$ are function spaces, enables mesh-free, discretization-agnostic learning—sensor locations can differ between samples; loss and inference are pointwise and mesh-indifferent (Zhang et al., 2022).
• Mesh-free Topology Optimization (DMF-TONN): Joint Fourier-feature MLPs parameterize density and displacement fields; the full optimization is performed via Monte Carlo sampled loss over the domain, avoiding any FEA meshing (Joglekar et al., 2023).
• High-dimensional Manifold Approximation: Approaches using Hermite/Gaussian kernels, or RVFL/Fredholm frameworks, yield function approximations on data-defined or embedded manifolds with no eigen-decomposition or atlas construction, and mesh-free sampling (Mhaskar, 2019, Needell et al., 2020, Gelß et al., 2023).

5. Advantages, Limitations, and Practical Considerations

Advantages

• Mesh-independence: All sampling, loss, and inference proceed from non-meshed, sampled points; implementation is simplified, and no geometric preprocessing is required (Ngom et al., 2020, Needell et al., 2020, Joglekar et al., 2023, Zhang et al., 2022).
• Universal function/operator approximation: Theoretical guarantees hold for wide classes of input/output spaces and for functional/operational fitting (Galimberti, 2024, Zhang et al., 2022, Yang et al., 2022).
• Geometric and topological generality: Non-metric and infinite-dimensional input spaces (quasi-Polish, Fréchet, Banach, data-defined manifolds) are admissible (Galimberti, 2024, Mhaskar, 2019).
• Dimension-independence: Approximation error and network size can be independent of domain dimension (with suitable smoothness/spectral conditions) (Thom, 2019, Yang et al., 2022).
• Interpretability: For trigonometric and integral-based architectures (e.g. FNNs, XNet), learned weights correspond to frequencies, amplitudes, and phases or localized kernels (Ngom et al., 2020, Li et al., 31 Jan 2025).

Limitations

• Spectral bias and precision limitations: Standard MLPs and feedforward architectures display bias toward lower frequencies; achieving true high-order convergence in mesh-free Lagrangian PDEs is often limited by inability to fully resolve high-frequency weights or solve ill-conditioned linear systems to residuals $<10^{-7}$ (Starepravo et al., 29 Mar 2025).
• Parameter efficiency and scalability: Large widths or high-order bases may be required for high-dimensional or extremely smooth targets; strategies such as basis learning (BelNet), Cauchy kernel stacking (XNet), or tensor-train compression (Fredholm nets) are essential for practical deployment (Gelß et al., 2023, Zhang et al., 2022, Li et al., 31 Jan 2025).
• Boundary and interface handling: While mesh-free approaches handle bulk domains well, treatment of complex boundaries, interfaces, or non-periodic boundary conditions may require auxiliary terms in the loss or architectural augmentations (window functions, piecewise subdivision, etc.) (He et al., 2020, Ngom et al., 2020, Bose et al., 2021).

Practical Implementation

Parameter and hyperparameter tuning, adaptive sampling, and coupling with physical constraints (physics-informed loss, hard masks, etc.) are common strategies to enhance performance, stability, and convergence (Joglekar et al., 2023, He et al., 2020, Zhang et al., 2022). Backpropagation and stochastic optimization algorithms are fully compatible with mesh-free parameterizations, including infinite-dimensional projections (Galimberti, 2024).

6. Controversies and Open Challenges

Unresolved challenges in mesh-free neural function approximation include:

• Achieving high-order convergence in high-order mesh-free PDEs: Despite accurate weight surrogates, neural networks have not yet replaced direct solvers at the $\gtrsim 10^{-7}$ residual tolerance required for Lagrangian high-order mesh-free methods; small high-frequency errors lead to divergence as grid spacing decreases (Starepravo et al., 29 Mar 2025).
• Universal approximation theorems for arbitrary operator learning: Rigorous extension to discretization-invariant, sample-independent operator UAPs is ongoing (e.g., full proof for mesh-free BelNet is open) (Zhang et al., 2022).
• Scalability in high dimensions or infinite-dimensional settings: While theoretical rates can be dimension-free, practical computation remains limited by the curse of dimensionality unless further structure (sparsity, tensorization) is exploited (Gelß et al., 2023, Mhaskar, 2019, Galimberti, 2024).

Potential remedies involve hybridizing mesh-free neural methods with physics constraints, exploring advanced architectures (transformers, SIREN/Fourier nets), adaptive sampling, and efficient surrogates for linear solvers with guaranteed stability (Starepravo et al., 29 Mar 2025, Li et al., 31 Jan 2025, Zhang et al., 2022).

7. Comparative Summary

Mesh-free neural function approximators constitute a theoretical and practical framework that decouples neural network-based regression, interpolation, and operator learning from the limitations imposed by classical mesh- or grid-based numerical schemes. These architectures accommodate general input geometries, high-dimensional or even infinite-dimensional data, provide universal (and often dimension-independent) approximation guarantees, and are applicable in pointwise PDE solving, scientific machine learning, and data-driven operator regression. They achieve this via a combination of universal (typically spectral) architectures, mesh-agnostic training protocols, and adaptive, physics-informed sampling and loss construction, though high-order and high-frequency accuracy in certain physical simulations remains a demanding open challenge (Ngom et al., 2020, Galimberti, 2024, Bose et al., 2021, Zhang et al., 2022, Li et al., 31 Jan 2025, Starepravo et al., 29 Mar 2025).