Implicit Neural Representation

• Implicit Neural Representation is a method where MLPs map spatial or temporal coordinates to continuous signal values, enabling resolution-agnostic and fully differentiable modeling.
• It integrates specialized activations, positional encodings, and hybrid strategies to overcome spectral bias and accurately capture high-frequency details.
• Practical implementations improve performance in tasks like image reconstruction, signal compression, and physics-informed inverse problems through efficient kernel-space transformations.

Implicit neural representations (INRs) are neural network parameterizations of continuous signals whereby a coordinate-based function—typically a multilayer perceptron (MLP)—maps spatial, spatiotemporal, or abstract coordinates directly to target signal values. This approach provides resolution-agnostic, differentiable, and highly compact representations, which have become central to contemporary signal processing, vision, inverse problems, and generative modeling. INRs underpin advances in areas such as image representation, geometric modeling, audio synthesis, and high-level vision tasks; they have motivated deep theoretical and architectural research to address foundations, limitations, and practical implementation challenges.

1. Core Formulation and Definitions

An INR models any continuous field $y$ as a neural function

$$f(\mathbf{x};\theta):\,\mathbb{R}^D \to \mathbb{R}^C$$

where $\mathbf{x}$ is a coordinate (pixel location, 3D position, time, hybrid spatio-angular features, etc.) and $\theta$ are network weights. Fitting proceeds via $\ell^2$ loss minimization over a discrete sample $\{\mathbf{x}_i,\mathbf{y}_i\}_{i=1}^N$: $$\theta^* = \arg\min_\theta \sum_{i=1}^N \|f(\mathbf{x}_i;\theta)-\mathbf{y}_i\|^2_2$$ Key mathematical properties include:

• Continuity: INRs interpolate naturally between sampling points and are not tied to a discretization scale.
• Full differentiability: Soft, smooth activations allow for arbitrary-order spatial derivatives, critical for applications involving PDE constraints or signal processing on the latent field (Essakine et al., 6 Nov 2024, Xu et al., 2022).
• Adaptive capacity: Width, depth, and underlying activation/encoding parameterization directly control representational power, with theoretical frameworks such as the neural tangent kernel (NTK) relating initialization and convergence characteristics to harmonic structure (Yüce et al., 2021, Ko et al., 19 Aug 2025).

2. Architectural Variants and Representation Taxonomy

Recent surveys (Essakine et al., 6 Nov 2024) and comparative studies systematize INR methods into four major classes:

Category	Approach	Principal Example(s)
Activation	Specialized nonlinearities	SIREN (sin), Wire (wavelet), Gauss (Gaussian), FINER (adaptive frequency), HOSC (sharp periodic), Sinc
Positional Encoding	Coordinate pre-processing	Fourier features, random Fourier maps, multi-scale projections
Combined strategies	Multi-part hybridization	Trident (Fourier→Gaussian), FLAIR (RC-Gauss + frequency encoding)
Structure	Network or layerwise innovations	INCODE (dynamic sinusoids), MFN (multiplicative filters), Fr (Fourier basis), hierarchical stacking (INRN)

Activation-centric INRs use periodic or spatially localized kernels to break the spectral bias of vanilla MLPs, which otherwise favor low-frequency content (see Section 3). The SIREN architecture employs scaled sine nonlinearities, enabling direct representation of high-frequency signals, while Gabor and wavelet-inspired activations combine spatial and frequency localization (Ko et al., 19 Aug 2025, Roddenberry et al., 2023).

Encoding-centric INRs map inputs through fixed or random orthogonal/frequency bases before feeding them to conventional activations. This augments the representational bandwidth without increasing MLP size, but choices of scale and distribution must be matched to the task.

Hybrid and combined approaches (e.g., FLAIR, Trident) explicitly unite frequency selectivity, space-frequency localization, and region-adaptive input modulation, often combining wavelet transforms or frequency guides with specialized activations (RC-GAUSS in FLAIR) (Ko et al., 19 Aug 2025).

Structural optimizations include dynamically conditioned activations, modularization, cross-layer harmonization, compressive and sparse architectures (Meta-SparseINR), and stacked hybrid blocks for both deep MLPs and deep conv–MLP hybrids (INRN) (Song et al., 2022, Lee et al., 2021).

3. Foundations: Harmonic Expansions, Spectral Bias, and Dictionary Analysis

INRs are fundamentally structured dictionaries of harmonics. In the analytically tractable case (sinusoidal activations or Fourier-feature MLPs), the representable signal is an integer linear combination of base frequencies. The bandwidth increases exponentially with network depth and polynomial activation order, while parameters grow linearly (Yüce et al., 2021). The NTK framework clarifies inductive bias: the eigenfunctions of the empirical kernel coincide with the dictionary atoms (harmonic components), and meta-learning (e.g., MAML) reshapes the kernel to fit the target class, enhancing signal alignment and fast adaptation.

A central limitation is spectral bias [Rahaman et al. 2019, (Essakine et al., 6 Nov 2024)]: standard MLPs converge rapidly to low-frequency signals, requiring higher capacity or architectural intervention to capture high-frequency detail. This motivates the use of periodic or wavelet activations, hierarchical depth (exponential harmonic expansion), explicit spatial/frequency guidance, and learned reparameterizations (Roddenberry et al., 2023, Zheng et al., 7 Apr 2025, Ko et al., 19 Aug 2025).

4. Kernel-Space Transformations and Coordinate Mappings

Beyond changing INR internals, transformation of input and output domains—kernel-space design—is effective for improving performance:

• Linear input scaling and output shifting (SS-INR, (Zheng et al., 7 Apr 2025)): Scaling input coordinates expands effective receptive bandwidth; output shifting centers signal values, acting as a form of normalization. These transformations boost PSNR by >6 dB with <1% compute overhead, effectively acting as zero-parameter "pseudo-layers" (Table 1 below).
• Coordinate reparameterization via hash embeddings (DINER, (Xie et al., 2022, Zhu et al., 2023)): Learnable hash tables reorder sample coordinates, restructuring the empirical spectrum so the subsequent MLP operates over a "smoother" representation. This nullifies the negative effects of coordinate disorder and spectral bias, dramatically accelerating convergence and enhancing high-frequency modeling.

Method (Kodim/SIREN)	PSNR (dB)	SSIM
Baseline	33.58	0.9202
Input scale only (s=5)	38.32	0.9671
Output shift only (b)	35.16	0.9402
Scale+Shift (SS-INR)	39.85	0.9758

Hash-embedding approaches like DINER show even larger absolute PSNR gains on 2D/3D signals, with memory overhead determined by hash width L and index size N. The expressive power saturates at L equal to the data rank (Xie et al., 2022, Zhu et al., 2023).

5. Domain-Specific and Advanced Applications

Signal Compression and Data-Driven Priors

INRs have shown competitive performance as signal compressors, especially when augmented by quantization, quantization-aware retraining, and meta-learned initialization. Compression via MAML-based adaptation expedites convergence and yields favorable rate–distortion curves compared to classical codecs and deep learned autoencoders. For images, shapes, and even 3D volumes, entropy-coded weight updates can compactly represent the signal with negligible PSNR loss (Strümpler et al., 2021, Lee et al., 2021).

Physics-Informed and Inverse Problem Solvers

The INR formalism applies naturally to PDE-constrained inverse problems. Level-set MLPs encode domains or obstacles as SDFs, facilitating automatic differentiation for shape optimization, mesh-free boundary integral evaluation, and convenient shape perturbation. Joint training with data-driven hypernetworks for priors (e.g., SDF manifolds) further regularizes ill-posed problems such as inverse scattering (Vlašić et al., 2022).

Time Series, Semantic, and High-Level Vision Tasks

Time-series modeling with INR architectures (SIREN, hypernetworks, frequency-domain losses) provides accurate, smoothly imputable representations and competitive generative models (HyperTime) (Fons et al., 2022). Extensions to semantic tasks include embedding global priors (e.g., via pre-trained feature extractors) directly into the INR weight tensors, enabling image fitting, view synthesis, and medical reconstruction to inherit domain-specific inductive biases (SPW, (Cai et al., 6 Jun 2024)). Generalizations of the INR formalism to high-dimensional, semantically rich signals (INRN) allow for classification, detection, and segmentation in a unified coordinate-based framework (Song et al., 2022).

Signal Processing Directly on INRs

Differentiable operators enable direct manipulation of INRs for signal processing purposes. INSP-Net reformulates linear convolutional (and some nonlinear) operations as polynomial combinations of high-order spatial derivatives, implementable via automatic differentiation on the INR's computational graph (Xu et al., 2022). This allows for blurring, denoising, filtering, and even classification to be handled natively in the implicit domain, circumventing discretization artifacts.

6. Challenges, Theoretical Limits, and Open Directions

Key technical and conceptual challenges in INR research include:

• Hyperparameter sensitivity: Frequency scales, band-limits, hash-table width, and depth must be carefully tuned to the data and application (Essakine et al., 6 Nov 2024).
• Scalability: High-dimensional signals or massive datasets necessitate efficient memory management (e.g., Meta-SparseINR, hash grids) and fast adaptation mechanisms (e.g., INT, (Zhang et al., 17 May 2024)).
• Generalization to unseen coordinates: Discrete hash-based approaches (DINER) require further innovation for interpolating in non-gridded domains or learning continuous coordinate-to-feature mappings.
• Expressivity vs. overfitting: Dynamic and high-capacity activations risk overfitting, especially on small or noisy datasets (Essakine et al., 6 Nov 2024).
• Quantum extension: Quantum neural networks (QIREN) achieve exponential Fourier spectrum growth in theory, with initial practical results in image fitting and superresolution. Quantum resource requirements, noise, and hybridization with classical layers are active research topics (Zhao et al., 6 Jun 2024).

Emergent future themes include: learned or adaptive kernel transformations, continuous semantic augmentation, combined local–global structure induction, theoretical characterization of non-classical activations, real-time and robust INR pipelines, and quantum-classical INR architectures.

7. Empirical Performance and Practical Considerations

Benchmarking across diverse domains reveals:

• Spectral bias is universally mitigated by periodic and wavelet activations, kernel-space reparameterizations, and semantic priors. Methods such as FLAIR, SS-INR, DINER, and SPW consistently achieve multi-decibel improvements in PSNR and perceptual metrics, often with negligible parameter and computational overhead.
• INRs remain competitive with or superior to classical compressed sensing, PDE solvers, and learned codecs in inverse problems, superresolution, compression, and generative modeling, especially when advanced guidance mechanisms are leveraged (Ko et al., 19 Aug 2025, Paz et al., 3 Feb 2024, Cai et al., 6 Jun 2024).
• Plug-and-play efficiency gains (INT) and model size reductions (Meta-SparseINR) do not compromise signal fidelity, and are readily combined with other INR innovations.

Both theoretical and empirical evidence position INRs as foundational to future research in continuous, compact, differentiable, and semantically-adaptive representation of signals across scientific, engineering, and creative fields.