Layered Implicit Neural Representations

Updated 10 July 2026

Layered implicit neural representations are INR models that use explicit architectural layers or modular decompositions to partition signal functions.
They employ diverse designs such as Split-Layer, hierarchical multilevel structures, and shared/private decompositions to enhance feature expressivity and manage computational complexity.
Empirical evidence shows that these methods improve reconstruction accuracy and parameter efficiency across tasks like image fitting, CT reconstruction, and novel view synthesis.

Searching arXiv for the cited INR papers and closely related layered/modular INR work. Layered implicit neural representations are implicit neural representations (INRs) whose representational power is organized through explicit architectural layering, modular decomposition, or layer-specific function roles rather than through a single monolithic coordinate MLP alone. In the INR setting, a signal is modeled as a continuous function from coordinates to attributes, but layered variants alter how that function is built: by replacing standard dense hidden layers with structured interaction layers, decomposing a signal into multiple implicit subfunctions, splitting low-pass and high-pass content across subnetworks, or separating shared and instance-specific roles across depth. Across recent work, this idea appears in several technically distinct forms: architectural layer reformulation for higher-order feature interactions (Cai et al., 13 Nov 2025), progressive multi-level residual INRs (Landgraf et al., 2022), explicit two-stream decomposition for layer separation in burst imagery (Nam et al., 2021), shared/private layer factorization across images (Zhou et al., 19 Aug 2025), and wavelet-structured coarse-to-fine implicit fields (Roddenberry et al., 2023). A related but different line defines neural layers implicitly through equations or optimization conditions rather than explicit feed-forward maps (Zhang et al., 2020).

1. Definition and scope

In the standard coordinate-based formulation, an INR models a signal as a continuous function $f(\mathbf{x};\theta)$ mapping coordinates $\mathbf{x}$ to target attributes $\mathbf{y}$ . A representative MLP form is

$\begin{aligned} \Vec{z}^{\:0} &= \Vec{x}\ \Vec{z}^{\:l}&=\mathbf{W}^l\Vec{z}^{\:l-1}+\Vec{b}^{\:l-1},\ \ l=1\dots L \ \Vec{z}^{\:l}_{act}&=\rho(\Vec{z}^{\:l})\ f(\Vec{x};\theta) &= \Vec{z}^{\:L}_{act}. \end{aligned}$

This baseline underlies much of INR research, including image fitting, shape representation, inverse problems, and radiance-field modeling (Cai et al., 13 Nov 2025).

Within this general framework, “layered” can refer to several non-equivalent design choices. One meaning is architectural: the hidden layer itself is redesigned, as in Split-Layer, where a dense layer is replaced by parallel branches combined by a Hadamard product (Cai et al., 13 Nov 2025). Another meaning is functional decomposition: different implicit functions represent different components of a signal, as in two-stream neural image representations for scene and interference layers (Nam et al., 2021). A third meaning is hierarchical composition across scales, where separate INR modules progressively refine a representation from coarse structure to fine detail (Landgraf et al., 2022). A fourth meaning is shared/private factorization across multiple signals, where some layers are shared globally and others remain signal-specific (Zhou et al., 19 Aug 2025). A fifth meaning appears in wavelet-based INRs, where the first layer supplies localized template atoms and deeper layers algebraically refine them into higher-frequency structure (Roddenberry et al., 2023).

A different but conceptually adjacent usage appears in the implicit-layers literature, where a neural layer is defined by a condition such as $F(y^{(k)},y^{(k+1)})=0$ rather than by an explicit formula $y^{(k+1)}=f(y^{(k)})$ (Zhang et al., 2020). This concerns the definition of a neural layer itself, not the use of an INR as a coordinate field, but it broadens the notion of what a “layer” in a layered neural representation can be.

2. Hidden-layer expressivity as a design target

A central architectural argument in recent INR work is that conventional dense MLP hidden layers constrain the dimensionality of the internal feature space. In the Split-Layer formulation, the $i$ -th preactivation of a standard dense layer is

$z_i^l = w_{i1}z_1^{l-1} + w_{i2}z_2^{l-1} + \cdot\cdot\cdot + w_{iC}z_C^{l-1},$

so each output is a linear combination of $C$ previous features, and the feature space is treated as

$\mathcal{F}^{C}=\mathcal{S}(z_1^{l-1},z_2^{l-1},\cdot\cdot\cdot,z_C^{l-1}).$

On this view, vanilla MLP feature-space dimensionality scales only linearly with hidden width $\mathbf{x}$ 0, whereas the parameter count and multiply-add cost of a dense $\mathbf{x}$ 1 layer scale quadratically (Cai et al., 13 Nov 2025).

Split-Layer addresses this by dividing a layer into $\mathbf{x}$ 2 parallel branches of width $\mathbf{x}$ 3, each with its own linear map, and merging branch outputs via elementwise Hadamard product: $\mathbf{x}$ 4 Expanding the $\mathbf{x}$ 5-th output shows that the resulting feature is a sum of degree- $\mathbf{x}$ 6 monomials of previous-layer features: $\mathbf{x}$ 7 The stated interpretation is that a split-layer spans a feature space whose dimensionality grows combinatorially as

$\mathbf{x}$ 8

while keeping hidden-layer parameter count approximately at $\mathbf{x}$ 9 by choosing branch widths $\mathbf{y}$ 0 (Cai et al., 13 Nov 2025).

This layer-level expressivity argument differs from techniques such as positional encoding, periodic activations, wavelets, or hash encodings. In the Split-Layer framing, those methods primarily alter inductive bias or coordinate parameterization, whereas the hidden-layer mechanism remains a conventional dense affine map. Split-Layer therefore presents layered INR design as a more fundamental intervention at the operator level (Cai et al., 13 Nov 2025).

A complementary theoretical perspective appears in the structured-dictionary view of INRs. For INRs with sinusoidal input mapping

$\mathbf{y}$ 1

and polynomial activations, the representable function class can be written as

$\mathbf{y}$ 2

with

$\mathbf{y}$ 3

This implies that harmonic order can grow as $\mathbf{y}$ 4, so depth expands frequency support exponentially while parameter count grows only linearly with depth (Yüce et al., 2021). In that view, layering is a recursive harmonic-generation mechanism rather than merely a stack of nonlinear transforms.

3. Major architectural patterns

Layered INRs currently appear in several recurring architectural forms.

First, there are layer-operator redesigns. Split-Layer replaces each hidden fully connected layer with a multi-branch multiplicative interaction module and is applied as a general hidden-layer substitute in ReLU MLP, SIREN, Gauss, PEMLP, WIRE, FINER, and NeRF-family backbones (Cai et al., 13 Nov 2025). SL $\mathbf{y}$ 5A-INR instead creates a stage-wise architecture in which a single learnable Chebyshev-parameterized activation layer processes coordinates first,

$\mathbf{y}$ 6

and deeper ReLU layers repeatedly fuse that first-layer output through elementwise multiplication: $\mathbf{y}$ 7 This assigns different roles to early and later layers: frequency-rich coordinate transformation first, stable ReLU feature mixing afterward (Heidari et al., 2024).

Second, there are hierarchical multilevel INRs. PINs partitions Fourier features into ordered subsets $\mathbf{y}$ 8 and builds the scene as

$\mathbf{y}$ 9

with higher levels receiving both their own band-limited encoding and previous-level latent features: $\begin{aligned} \Vec{z}^{\:0} &= \Vec{x}\ \Vec{z}^{\:l}&=\mathbf{W}^l\Vec{z}^{\:l-1}+\Vec{b}^{\:l-1},\ \ l=1\dots L \ \Vec{z}^{\:l}_{act}&=\rho(\Vec{z}^{\:l})\ f(\Vec{x};\theta) &= \Vec{z}^{\:L}_{act}. \end{aligned}$0 Each prefix of the hierarchy yields a valid continuous representation at a coarser level of detail (Landgraf et al., 2022).

Third, there are explicit multi-function decompositions. In neural image representations for multi-image fusion and layer separation, a canonical scene INR and a nuisance/interference INR are optimized jointly: $\begin{aligned} \Vec{z}^{\:0} &= \Vec{x}\ \Vec{z}^{\:l}&=\mathbf{W}^l\Vec{z}^{\:l-1}+\Vec{b}^{\:l-1},\ \ l=1\dots L \ \Vec{z}^{\:l}_{act}&=\rho(\Vec{z}^{\:l})\ f(\Vec{x};\theta) &= \Vec{z}^{\:L}_{act}. \end{aligned}$1 The observed image is then formed either additively,

or through task-specific compositing such as fence alpha compositing or rain compositing (Nam et al., 2021). NIVeL applies a related idea to 2D vector-like graphics, representing an image as ordered implicit occupancy layers

with final rendering

so the representation behaves like editable stacked vector layers (Thamizharasan et al., 2024).

Fourth, there are shared/private decompositions across signals. MINR keeps the input layer, an added projection layer, and the output layer private to each image, while sharing the intermediate layers across images. For image $\begin{aligned} \Vec{z}^{\:0} &= \Vec{x}\ \Vec{z}^{\:l}&=\mathbf{W}^l\Vec{z}^{\:l-1}+\Vec{b}^{\:l-1},\ \ l=1\dots L \ \Vec{z}^{\:l}_{act}&=\rho(\Vec{z}^{\:l})\ f(\Vec{x};\theta) &= \Vec{z}^{\:L}_{act}. \end{aligned}$5,

The layered premise is that early and late layers are image-specific, whereas middle layers learn reusable transformations (Zhou et al., 19 Aug 2025). A related generalization-oriented design modulates only one low-rank weight block in an early layer, $F(y^{(k)},y^{(k+1)})=0$ 0, while all later layers remain shared as instance-agnostic composition rules (Kim et al., 2022).

Fifth, there are wavelet-structured layered INRs. In the complex-wavelet formulation,

$F(y^{(k)},y^{(k+1)})=0$ 1

and deeper analytic nonlinearities generate polynomial combinations of first-layer localized atoms. The paper advocates a split architecture with a shallow scaling INR for smooth low-pass content and a deeper Gabor INR for nonsmooth high-pass content, whose outputs are summed (Roddenberry et al., 2023).

4. Functional interpretations of layers

Across these designs, a recurring theme is that depth is assigned explicit representational roles rather than treated as homogeneous repetition.

One interpretation is coarse-to-fine spectral refinement. PINs assigns low-frequency Fourier bands to early levels and progressively higher bands to later ones, with additive residual composition (Landgraf et al., 2022). The complex-wavelet analysis states that first-layer atoms provide coarse localized approximations, while later layers generate higher-order coupled frequency content through products of those atoms, thereby recovering finer structure around singularities (Roddenberry et al., 2023). SL $F(y^{(k)},y^{(k+1)})=0$ 2A-INR similarly privileges the first layer as the locus of rich spectral transformation and uses deeper ReLU layers mainly for nonlinear fusion (Heidari et al., 2024).

A second interpretation is local-global factorization. Multi-head ReLU INR networks use a shared body $F(y^{(k)},y^{(k+1)})=0$ 3 and local rendering heads $F(y^{(k)},y^{(k+1)})=0$ 4 so that

$F(y^{(k)},y^{(k+1)})=0$ 5

The body learns global features of the coordinate-to-signal mapping, while heads reconstruct disjoint local regions of the signal. The stated rationale is that each head sees a locally simpler, effectively lower-frequency subproblem, mitigating spectral bias in ReLU coordinate networks (Aftab et al., 2021).

A third interpretation is content decomposition. In burst-image layer separation, one stream models the desired scene under motion-consistent canonicalization and another models interference unconstrained by the scene-motion prior (Nam et al., 2021). In NIVeL, separate implicit occupancy layers correspond to ordered filled regions with per-layer colors, making the representation editable and vector-like (Thamizharasan et al., 2024). In 1D magnetotelluric inversion, the representation is continuous in depth but sampled into layered physics for the forward solve, so the “layered” aspect emerges at the interface between neural parameterization and classical layered recursion rather than in the network architecture itself (Shaik et al., 5 Jun 2026).

A fourth interpretation is role specialization across depth. In the instance pattern composer framework, the first shared layer extracts common frequency patterns, the modulated second layer composes instance-specific content patterns,

$F(y^{(k)},y^{(k+1)})=0$ 6

and later layers act as shared composition rules (Kim et al., 2022). MINR reports a parallel finding across separately trained image INRs: first and last layers vary across images, while intermediate layers follow similar bell-shaped distribution patterns and can be shared effectively (Zhou et al., 19 Aug 2025).

A fifth interpretation concerns operator learning over implicit fields rather than signal representation alone. INSP-Net defines a processed implicit field by combining a given INR and its derivatives: $F(y^{(k)},y^{(k+1)})=0$ 7 Its layered specialization, INSP-ConvNet, stacks derivative-fusion blocks so that intermediate activations remain continuous feature fields rather than grid tensors (Xu et al., 2022). This suggests that layered INRs can also be understood as networks whose hidden “feature maps” are themselves implicit functions.

5. Empirical evidence across tasks

Empirical results indicate that layered designs are useful in several INR regimes, especially when hidden-layer expressivity or structured decomposition is limiting.

Split-Layer reports gains on four tasks: 2D image fitting, 2D CT reconstruction, 3D shape representation, and 5D novel view synthesis (Cai et al., 13 Nov 2025). In 2D image fitting on 16 natural images at $F(y^{(k)},y^{(k+1)})=0$ 8, with 4 hidden layers of width $F(y^{(k)},y^{(k+1)})=0$ 9, 50,000 Adam iterations, and PSNR as the metric, ReLU improves from 21.24 dB to 30.89 dB, Gauss from 31.74 dB to 40.84 dB, PEMLP from 29.60 dB to 40.78 dB, and FINER from 38.72 dB to 40.09 dB. In sparse-view 2D CT reconstruction with 100 emulated measurements, SIREN improves from 18.32 dB to 29.11 dB and PEMLP from 28.11 dB to 32.29 dB. In 3D occupancy representation, Split-PEMLP reaches $y^{(k+1)}=f(y^{(k)})$ 0 Chamfer distance, the best among compared variants. Novel-view-synthesis gains are smaller but consistent, with NeRF moving from 29.41 to 30.20 average PSNR (Cai et al., 13 Nov 2025).

PINs reports improved reconstruction accuracy, representational capacity, and training speed relative to baselines on 2D and 3D tasks (Landgraf et al., 2022). On ImageNet image regression, the full model obtains MSE 0.0027, PSNR 75.9, and VGG feature distance 1.89, compared with FF Net at 0.0048/74.5/2.06 and SIREN at 0.015/72.1/2.36. On DIV2K it reports $y^{(k+1)}=f(y^{(k)})$ 1 MSE and 76.8 PSNR, slightly ahead of SAPE at $y^{(k+1)}=f(y^{(k)})$ 2 and 76.5. In 3D shape regression, PINs is often best or tied-best in bidirectional Chamfer distance across categories such as lamp, chair, and camera (Landgraf et al., 2022).

For explicit layered decomposition, neural image representations for multi-image fusion and layer separation report that reference-free canonical fusion plus two-stream decomposition supports moiré, reflection, fence, and rain removal (Nam et al., 2021). In synthetic moiré removal, the method attains PSNR 38.68 and SSIM 0.9751, with a supplementary comparison showing that replacing learned motion with conventional homography estimation and a center-frame reference drops PSNR from 38.68 to 23.94. In rain removal, adding the occlusion coordinate $y^{(k+1)}=f(y^{(k)})$ 3 improves PSNR from 20.42 to 24.15 in the ablation (Nam et al., 2021).

NIVeL reports that representing images as implicit vector layers yields stronger text-to-vector quality than the compared explicit-curve baseline. At 1K parameters, VectorFusion scores R-Prec 59.5 and Sim 21.4, while NIVeL scores 68.2 and 25.1. At 12K parameters, VectorFusion reaches 71.3 and 26.7, while NIVeL reaches 78.5 and 32.0 (Thamizharasan et al., 2024).

MINR demonstrates that shared/private layer decomposition yields large parameter savings for multi-image encoding. On 4-image Kodak reconstruction, separate SIRENs use 0.80M parameters and achieve $y^{(k+1)}=f(y^{(k)})$ 4 dB, whereas MINR uses 0.47M parameters and attains $y^{(k+1)}=f(y^{(k)})$ 5 dB. On 100 ImageNet images, separate SIRENs use 19.88M parameters, whereas MINR uses 6.93M and still maintains $y^{(k+1)}=f(y^{(k)})$ 6 dB PSNR and $y^{(k+1)}=f(y^{(k)})$ 7 SSIM (Zhou et al., 19 Aug 2025).

The instance pattern composer design reports strong cross-instance generalization. On $y^{(k+1)}=f(y^{(k)})$ 8 images, it surpasses TransINR on CelebA, FFHQ, and ImageNette, with especially large gains on ImageNette. Its ablation over modulation location shows that modulating the second layer is best: ImageNette PSNR is 35.93 for layer 2, versus 31.00 for layer 1 and 20.26 for layer 5 (Kim et al., 2022). This supports the claim that different depths have different functional roles.

6. Limitations, misconceptions, and open directions

A common misconception is that “layered INR” names a single architectural family. The literature instead uses the term implicitly across several design axes: hidden-layer reformulation, explicit content separation, multilevel residual hierarchies, shared/private parameter factorizations, and even operator stacks over implicit fields. These approaches are related by their use of layer structure as a representational prior, but they solve different problems and impose different inductive biases (Cai et al., 13 Nov 2025, Landgraf et al., 2022, Nam et al., 2021).

Another misconception is that increased formal feature-space dimensionality or harmonic support directly guarantees better approximation or optimization. Split-Layer explicitly presents a constructive dimensionality argument rather than a classical approximation theorem and notes that increasing the split count $y^{(k+1)}=f(y^{(k)})$ 9 shrinks branch width, so larger formal polynomial degree need not monotonically improve performance (Cai et al., 13 Nov 2025). The structured-dictionary perspective likewise separates expressivity from learnability: depth expands the harmonic dictionary, but the neural tangent kernel determines which atoms are actually easy to learn (Yüce et al., 2021).

A further misconception is that continuous neural parameterization removes inverse-problem nonuniqueness. In 1D magnetotelluric inversion, the INR represents resistivity continuously in depth,

$i$ 0

but the forward model still samples this field onto a layered grid and applies Wait’s recursive impedance formulation. The method does not recover “true layers” uniquely; rather, it often returns a smooth representative within the MT equivalence class, especially for thin or deep conductors (Shaik et al., 5 Jun 2026). This suggests that layered INRs do not eliminate physical ill-posedness; they change how ambiguity is expressed.

Several open directions recur across the literature. One is sharper interface modeling. The MT inversion study notes that a standard LeakyReLU INR with midpoint sampling tends toward smooth approximations of layered Earth and suggests that sharper recovery may require richer coordinate encodings, periodic activations, piecewise or hybrid implicit functions, adaptive sampling, or explicit interface-aware priors (Shaik et al., 5 Jun 2026). Another is scalable modularity: MINR and instance-specific composer methods suggest that early and late layers may encode signal-specific behavior while middle layers are reusable, but the evidence remains mostly empirical and task-specific (Zhou et al., 19 Aug 2025, Kim et al., 2022). A third is continuous operator depth: INSP-ConvNet demonstrates a continuous convolution-like hierarchy over INRs, but higher-order derivatives are expensive and unstable, limiting deeper models (Xu et al., 2022). A fourth is dynamic compositionality: NIVeL fixes a maximum number of layers $i$ 1, and its vector extraction is post hoc rather than differentiable (Thamizharasan et al., 2024).

The broader survey literature frames these issues under network-structure optimization, emphasizing that improved layer design must balance spectral expressivity, spatial compactness, trainability, and computational cost (Essakine et al., 2024). That summary is consistent with the primary lesson of layered INR research: depth and modularity matter not simply because deeper networks are more expressive, but because the organization of computation across layers determines whether an INR behaves as a coarse-to-fine refiner, a local-global decomposer, a shared/private factorization, or a compositional signal model.