Coordinate-Based Neural Networks

• Coordinate-based neural networks are models that map spatial or temporal coordinates to continuous signal values using implicit representations.
• They employ multilayer perceptrons with advanced encoding techniques like Fourier features and multi-resolution hash encoding to control spectral bias and improve convergence.
• These networks enable rapid adaptation, efficient compression, and high-accuracy reconstructions in applications such as 3D modeling, image compression, and inverse problem solving.

A coordinate-based neural network is a neural architecture that represents complex signals as mappings from spatial, temporal, or other coordinate domains to continuous function values. The signal is modeled implicitly via parameterized fits, typically using multilayer perceptrons (MLPs), endowed with expressive encodings and occasionally meta-learning paradigms for rapid adaptation and efficient storage. Coordinate-based representations are central for implicit neural fields in images, 3D geometry, volumetric data, inverse problems, compression, and neural network meta-representations.

1. Mathematical Foundations and Core Architectures

Coordinate-based neural networks define a family of functions $f_\theta: \mathcal{X} \rightarrow \mathbb{R}^C$, where $\mathcal{X}\subseteq\mathbb{R}^d$ is the (spatial, spatiotemporal, discrete, or graph) coordinate domain and the output may be scalar or vector-valued (e.g., color, density, amplitude/phase, class logits). The canonical architecture is an MLP, such as

$$y = f_\theta(x) = \operatorname{MLP}_\theta(\operatorname{Encode}(x)),$$

with an application-dependent encoding of the input coordinate $x$.

A representative example is volumetric data compression, where each 3D coordinate $x\in[0,1]^3$ is mapped to an intensity $y\in\mathbb{R}$ by an MLP with ReLU activations, employing a multi-resolution hash encoding $\operatorname{Encode}(x)$ for increased fidelity: $$y = f_\theta(x) = W^{(3)}(\mathrm{ReLU}(W^{(2)} \mathrm{ReLU}(W^{(1)}\,\mathrm{Encode}(x) + b^{(1)}) + b^{(2)})) + b^{(3)}$$ with all weights, biases, and hash tables comprising $\theta$ (Devkota et al., 16 Jan 2024).

For many applications, sine or other periodic activations (SIREN, MFN, BACON), as well as expressive input encodings (Fourier features, multi-resolution hash, shifted bases), are preferred to control the spectral inductive bias and representation bandwidth (Lindell et al., 2021, Zheng et al., 2022, Devkota et al., 16 Jan 2024).

2. Encoding Schemes and Spectral Characteristics

Encoding the coordinate input plays a decisive role in the representation capacity and convergence properties of coordinate networks. Approaches include:

• Fourier Features / Random Fourier Embeddings: Lift $x$ to high-dimensional spaces with sinusoidal basis $\sin(2\pi B x)$ for a set of frequencies $B$, improving the representation of high-frequency structure. The performance is governed by the stable rank of the embedded coordinate matrix and the preservation of distance in the embedding (Zheng et al., 2022).
• Multi-Resolution Hash Encoding: Employed in volumetric data compression, $L$ grid levels of increasing resolution are laid over the domain; hash functions assign discrete feature vectors to grid corners, which are interpolated and concatenated to yield $\operatorname{Encode}(x)\in\mathbb{R}^{L\cdot W}$ (Devkota et al., 16 Jan 2024).
• Shifted/Generalized Basis Embeddings: Any family of shifted, bandlimited, continuous basis functions $\psi(t-x)$ may be used. With high embedding complexity (e.g., Gaussian, triangle), memorization is possible by a shallow or even linear model; with low complexity, smooth generalization is induced (Zheng et al., 2022).
• Multiplicative Filter Networks / BACON: Stacks of sinusoidal filters with frozen frequencies enforce a bandlimit, yielding an analytically tractable, strictly limited signal spectrum, and allow multiscale outputs per-layer (Lindell et al., 2021).

The inductive bias (the "spectral bias") induced by the choice of encoding and activation directly impacts the frequency components that the network can fit and the training speed for high-frequency versus low-frequency components (Lindell et al., 2021, Cai et al., 25 Jul 2024).

3. Addressing Spectral Bias via Architectural and Normalization Techniques

Coordinate-based MLPs exhibit "spectral bias": they fit low-frequency signal components rapidly and high-frequency components slowly or not at all, reflecting the spectrum of the neural tangent kernel (NTK)—typically, a few large eigenvalues and a long tail of small ones.

Recent work provides a precise NTK analysis revealing that standard coordinate MLPs' spectral bias stems from the distribution of the NTK eigenvalues (Cai et al., 25 Jul 2024). Batch normalization (BN), layer normalization (LN), and novel joint-scope normalizations—global normalization (GN) and cross normalization (CN)—reduce the variance and maximum eigenvalue of the NTK while preserving its mean, compressing the spectrum and shifting the bulk upward. Consequently, high-frequency components are learned more rapidly, and the overall convergence rate is improved. Empirically, applying CN to various architectures and encodings advances the state-of-the-art across image compression, computed tomography, 3D shape fitting, MRI reconstruction, NeRF view synthesis, and multi-view stereo (Cai et al., 25 Jul 2024).

Table: Key spectral statistics for MLPs and normalization-augmented MLPs

Method	Variance ($v_\lambda$) scaling	Max eigenvalue ($\lambda_{\max}$) scaling
Std MLP	$\mathcal{O}((N+2\sqrt N+C)TN)$	$\mathcal{O}(N\sqrt{C}T + \sqrt{N}CT)$
BN-MLP	$\mathcal{O}(CTN)$	$\mathcal{O}(N\sqrt{CT}+\sqrt{N}CT)$
LN-MLP	$\mathcal{O}((N+2\sqrt N+T)CN)$	$\mathcal{O}(NC\sqrt{T}+\sqrt{N}CT)$

Applying normalization transforms the coordinate MLP from a low-pass system to one with more balanced frequency learning—without requiring hand-tuned positional encodings.

4. Meta-Learning and Speed of Adaptation

Optimizing coordinate-based networks from scratch for each new signal is inefficient. Meta-learning (first-order: Reptile; second-order: MAML) trains the initial weights $\theta^*_0$ such that, for any observed signal from a target class, a few steps of (stochastic) gradient descent suffice for accurate encoding (Tancik et al., 2020, Devkota et al., 16 Jan 2024).

Formally, the meta-objective is

$$\theta^*_0 = \arg\min_{\theta_0} \mathbb{E}_{T\sim \mathcal{T}}\left[L(\theta_m(\theta_0, T))\right]$$

where $\theta_m$ is the adapted parameter after $m$ task-specific gradient steps and $L$ is the reconstruction or task loss.

Empirically, meta-initialized coordinate networks enable:

• Dramatic speed-up in convergence (order-of-magnitude reduction in fitting steps, e.g., first-100-step PSNR gain of 2–5 dB (Devkota et al., 16 Jan 2024));
• Stronger priors for under-constrained inverse problems (one-view 3D reconstruction, few-view CT/MRI, and appearance transfer) (Tancik et al., 2020);
• Parameter-efficient adaptation for volumetric or image compression (Devkota et al., 16 Jan 2024).

5. Applications and Scaling Strategies

Coordinate-based neural networks support a broad set of domains and tasks:

• Compression and Representation of Volumetric Data: Networks model $f_\theta:\mathbb{R}^3 \to \mathbb{R}$ with multi-resolution hash encoding and meta-learned initialization, achieving 100:1–500:1 compression ratios at 30–45 dB PSNR, outperforming frequency-based schemes in both compression and convergence (Devkota et al., 16 Jan 2024).
• Gigapixel and Complex Scene Fitting: Hybrid schemes, such as ACORN, partition the domain hierarchically (e.g., quad/octree), combining heavy shared encoders with lightweight decoders per block, scaling to 8K+ images and 3D shapes with fast (1.8–2.6 h for 64–996 MP) fitting and nearly 42 dB PSNR (Martel et al., 2021).
• Patchwise and Adversarial Internal Learning: Patch-based coordinate MLPs, such as Neural Knitwork, integrate overlapping local predictions with adversarial and consistency losses, matching CNN-based internal learning in inpainting, denoising, and super-resolution with 80% fewer parameters (Czerkawski et al., 2021).
• Graph Machine Learning: By assigning virtual or topology coordinates (via shortest paths or low-rank embeddings) to graph nodes, and training MLPs on these coordinates, TCNN and DVCNN architectures nearly match GCN/SAGE on OGBN-Products while requiring one or two orders of magnitude fewer parameters, with substantially reduced per-epoch cost (Qin et al., 2023).
• Fourier Phase Retrieval and Ptychography: Unsupervised coordinate-MLPs (e.g., SCAN) map normalized spatial coordinates to amplitude and phase, trained via spectral-domain losses. These models achieve superior phase recovery and robustness to noise over iterative and learning-based baselines (Li et al., 2023).
• Meta-Representations of Neural Networks: Assigning coordinate labels to each convolutional kernel, NeRN trains a mapping from these coordinates to kernel weights, reconstructing CNNs at 30–60% of the original model size, with negligible accuracy drop (Ashkenazi et al., 2022).
• Accelerated Architectures: Split-branch schemes (CoordX) learn each coordinate dimension separately in early layers and fuse representations, yielding 2–3× speedup in training and inference with marginal loss in fidelity, especially for highly-structured domains (images, videos, volumes) (Liang et al., 2022).

6. Multiscale, Bandlimited, and Hybrid Representations

Coordinate networks support multiscale and spectrum-constrained representations:

• BACON: By stacking multiplicative filter layers with fixed frequency support, BACON yields a model whose Fourier spectrum is analytically known and strictly bandlimited. Multi-head outputs after each filter provide level-of-detail control without per-scale supervision, ensuring correct aliasing characteristics at all scales (Lindell et al., 2021).
• Hybrid MLP–tensorial feature approaches: For neural radiance fields (NeRFs), combining a coordinate MLP (capturing low-frequency structure via ReLU bias) with multi-plane tensor representations (capturing high frequencies) achieves superior results—especially in settings with sparse data. Progressive channel engagement and residual integration yield PSNR and SSIM improvements over baseline methods under both static and dynamic conditions (Kim et al., 13 May 2024).
• Blockwise Implicit Representations: For data with complex or variable local geometry and attributes (e.g., point clouds), space is hierarchically partitioned and local coordinate-based MLPs are shifted to each block origin, optionally modulated by blockwise latent vectors, as in LVAC. This enables rate–distortion optimized compression, outperforming RAHT by 2–4 dB at typical bit rates (Isik et al., 2021).

7. Limitations, Practical Guidelines, and Outlook

Limitations and design considerations include:

• Spectral trade-offs: Choice of encoding and activation dictates accessible frequencies; pure ReLU or tanh MLPs tend to underfit fine detail; adaptive normalization or band-limiting methods are needed for high-fidelity multi-scale signals (Cai et al., 25 Jul 2024, Lindell et al., 2021).
• Scalability: Hierarchical domain partitioning (ACORN), blockwise representations (LVAC), or split MLPs (CoordX) are vital for tractable training/inference on very large or high-dimensional signals.
• Task-specific adaptation: Meta-learned initializations and task family-specific prior learning substantially accelerate convergence and regularization but require sufficient meta-training data and class homogeneity.
• Implementation: For signals aligned to separable grids, complex embedding + shallow linear networks drastically accelerate fitting versus deep MLPs with simple encodings (Zheng et al., 2022).
• Graph and weight-space coordinates: Assigning architectural or topological coordinates to non-Euclidean domains enables extending coordinate-based models to graphs, neural network weight compression, and beyond (Qin et al., 2023, Ashkenazi et al., 2022).

Extensions to differentiable resource allocation, spatiotemporal signals, graph coordinates for evolving data, joint compression of neural fields, and fully end-to-end block selection continue to be active research directions (Martel et al., 2021, Isik et al., 2021).

---

Coordinate-based neural networks have matured from simple MLPs trained per-signal to highly-structured, meta-learned, and normalization-enhanced systems, attaining state-of-the-art efficiency and accuracy in data compression, geometric modeling, large-scale inverse problems, and internal or cross-system representations. Their combination with advanced encoding, spectrum control, and learning-theoretic frameworks positions them as foundational in neural representation and machine perception theory and practice (Devkota et al., 16 Jan 2024, Cai et al., 25 Jul 2024, Lindell et al., 2021, Tancik et al., 2020).