Masked Implicit Neural Representations

• Masked Implicit Neural Representations (MINR) are a self-supervised framework that uses a continuous function to map coordinates to RGB values.
• It utilizes a transformer-based hypernetwork to predict an image-specific MLP, leading to robust in-domain and out-of-distribution reconstructions.
• MINR shifts the reconstruction paradigm from discrete token completion to continuous function estimation, delivering higher PSNR and parameter efficiency.

Searching arXiv for the cited MINR paper and closely related background papers. Masked Implicit Neural Representations (MINR) is a self-supervised framework that fuses implicit neural representations with masked image modeling. In MINR, an image is represented by a continuous function $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$, and a masked input image is processed by a transformer-based hypernetwork that predicts the weights $\theta$ of an image-specific implicit neural representation (INR). The resulting INR reconstructs the full image through continuous coordinate-to-pixel mapping rather than direct regression of discrete patch tokens. The framework is introduced as a response to limitations of standard masked autoencoders (MAE), especially their dependence on masking ratio, patch size, and arrangement, and their tendency to degrade when test-time masking deviates from training. Reported experiments indicate superior in-domain and out-of-distribution reconstruction, together with substantially lower parameter counts than MAE baselines (Lee et al., 30 Jul 2025).

1. Conceptual basis and motivation

MINR is motivated by two observations. First, standard masked autoencoders learn a discrete mapping from visible to masked patches and depend heavily on the exact masking ratio, patch size, and arrangement. Second, they compute loss only on masked tokens, which leads to poor generalization when the test-time masking deviates from training. By contrast, implicit neural representations learn a continuous function $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$ that maps any coordinate to its RGB value, yielding a single continuous mapping irrespective of how many or which patches are masked, and permitting queries at arbitrary resolutions or under unusual mask patterns (Lee et al., 30 Jul 2025).

The defining synthesis in MINR is three-stage: a large fraction of 2D patches is masked from the input image; the masked image is fed into a transformer-based hypernetwork that predicts the INR weights $\theta$; and the INR reconstructs the full image via continuous coordinate-to-pixel mapping. In the paper’s summary formulation, MINR learns to “fill in” missing pixels by first predicting an image-specific continuous MLP, rather than by directly regressing discrete patch tokens.

This design places the continuous function, rather than the patch token, at the center of representation learning. A plausible implication is that the framework shifts the reconstruction problem from token completion to per-instance function estimation, which aligns with the reported robustness to masking strategy and domain shift.

2. Functional formulation and reconstruction objective

In MINR, an image is represented as a continuous function

$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$

where for any $2$D coordinate $x=(x_1,x_2)\in[0,1]^2$, $f_\theta(x)\in[0,1]^3$ gives the RGB value at that location (Lee et al., 30 Jul 2025).

Before feeding $x$ into the MLP, MINR applies a sinusoidal positional encoding $\gamma:\mathbb{R}^2\to\mathbb{R}^{2D}$:

$\theta$0

The INR itself is a standard $\theta$1-layer MLP with hidden dimension $\theta$2:

$\theta$3

$\theta$4

and

$\theta$5

with

$\theta$6

where $\theta$7 is an element-wise nonlinearity such as ReLU or SiLU.

The masking operator is defined over the full image $\theta$8, arranged as a set of patch coordinates and values. A random masking operator $\theta$9 zeros out a fraction $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$0 of the patch values:

$f_\theta:\mathbb{R}^2\to\mathbb{R}^3$1

where each patch has probability $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$2 of being removed. After the hypernetwork predicts $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$3 from $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$4, the INR is evaluated at every patch center $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$5 and compared with the ground-truth value $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$6. The per-image $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$7 loss is

$f_\theta:\mathbb{R}^2\to\mathbb{R}^3$8

Averaging over the data distribution $f_\theta:\mathbb{R}^2\to\mathbb{R}^3$9 and random masks gives

$\theta$0

No extra regularizer is used beyond standard weight decay on $\theta$1.

A common misunderstanding is that masked-image frameworks necessarily supervise only the missing regions. In MINR, the reconstruction loss is computed at every patch center across the full image, not only on masked tokens. This distinction is central to the framework’s stated generalization behavior.

3. Training procedure and architectural variants

The training procedure fixes the masking protocol and optimization schedule. Images of resolution $\theta$2 are split into $\theta$3 patches, for a total of $\theta$4 patches. At each iteration, $\theta$5 of patches are randomly masked, corresponding to $\theta$6. There is no curriculum on mask ratio; training always uses $\theta$7 masking. Optimization uses AdamW with base learning rate $\theta$8, linear warmup over the first $\theta$9 of epochs, cosine decay thereafter, and a total of $f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$0 epochs of pretraining (Lee et al., 30 Jul 2025).

The hypernetwork is a ViT-style transformer encoder that ingests only the visible patches, that is, the unmasked tokens. It predicts INR weights in one of two variants. In TransINR, the hypernetwork predicts all INR layers. In GINR, only the second layer is instance-specific, while the remaining layers are shared.

The reported architectural specification is summarized below.

Component	Specification	Parameters
INR MLP	$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$1, hidden dimension $f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$2, sinusoidal positional encoding with $f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$3 frequencies	—
Hypernetwork	12-layer Vision Transformer encoder, as in ViT-Base	—
MINR-TransINR	Pooled output heads generate all $f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$4	$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$5M
MINR-GINR	Only layer 2 is instance-specific; remaining layers shared	$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$6M
MAE-Large	Baseline comparison	$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$7M
MAE-Base	Baseline comparison	$f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$8M

Within this setup, MINR is described as reducing model size by more than $f_\theta:\mathbb{R}^2\to\mathbb{R}^3,$9 while improving reconstruction quality. This suggests that the hypernetwork-plus-INR decomposition is not merely a change of decoder form; it is also a parameter-allocation strategy in which shared transformer processing and compact per-instance function prediction replace a much larger discrete reconstruction model.

4. Reconstruction performance in-domain and out-of-distribution

The reported evaluation uses PSNR in dB for in-domain reconstruction across CelebA, Imagenette, and MIT Indoor. MINR-TransINR and MINR-GINR both exceed MAE-Large and MAE-Base on all listed datasets (Lee et al., 30 Jul 2025).

Dataset	MAE-L / MAE-B	MINR-TransINR / MINR-GINR
CelebA	15.02 / 15.40	21.87 / 21.68
Imagenette	14.69 / 14.45	18.74 / 19.36
MIT Indoor	15.18 / 14.37	17.76 / 18.62

The paper states that MINR achieves approximately $2$0 dB PSNR gain on CelebA and more than $2$1 dB on the other datasets, despite having no more than $2$2M parameters.

The out-of-distribution transfer results likewise show a consistent advantage. For training-to-testing transfers CEL$2$3IMG, CEL$2$4IND, IMG$2$5CEL, IMG$2$6IND, IND$2$7CEL, and IND$2$8IMG, the MAE-Large and MAE-Base scores fall in the range of roughly $2$9 to $x=(x_1,x_2)\in[0,1]^2$0, whereas MINR-TransINR and MINR-GINR fall in the range of roughly $x=(x_1,x_2)\in[0,1]^2$1 to $x=(x_1,x_2)\in[0,1]^2$2. The paper summarizes this as MINR outperforming MAE by approximately $x=(x_1,x_2)\in[0,1]^2$3–$x=(x_1,x_2)\in[0,1]^2$4 dB under OOD, showcasing robust generalization across image domains.

These results are paired with ablations. A mask ratio of $x=(x_1,x_2)\in[0,1]^2$5 was optimal; lower or higher ratios degrade PSNR by $x=(x_1,x_2)\in[0,1]^2$6–$x=(x_1,x_2)\in[0,1]^2$7 dB. An MLP depth of $x=(x_1,x_2)\in[0,1]^2$8 layers outperformed shallower $x=(x_1,x_2)\in[0,1]^2$9-layer and deeper $f_\theta(x)\in[0,1]^3$0-layer networks by approximately $f_\theta(x)\in[0,1]^3$1 dB. GINR’s shared layers slightly boost OOD performance by $f_\theta(x)\in[0,1]^3$2 dB at small parameter cost.

Taken together, the quantitative pattern supports the paper’s claim that continuous reconstruction is less sensitive to the exact masking configuration than discrete token prediction. This suggests that the performance gain is not limited to in-domain memorization of a single masking regime.

5. Mechanistic interpretation and robustness profile

The paper identifies several conditions under which MINR excels: high mask ratios, novel mask patterns such as block masks and irregular shapes, and cross-domain reconstruction or few-shot domain shifts (Lee et al., 30 Jul 2025). The stated key insight is that learning a continuous mapping $f_\theta(x)\in[0,1]^3$3 reduces reliance on explicit neighboring patches. A second key insight is that hypernetwork weight modulation trades off per-instance fitting with shared structure.

These claims provide a mechanistic interpretation of the TransINR and GINR variants. TransINR maximizes per-instance flexibility by predicting all INR layers. GINR constrains part of the function class through shared layers and predicts only layer 2 per instance. The reported ablation that GINR slightly improves OOD PSNR by $f_\theta(x)\in[0,1]^3$4 dB is consistent with the stated trade-off between instance adaptation and shared structure.

A common misconception is that robustness in masked reconstruction is determined primarily by the masking strategy itself. The MINR formulation argues instead that robustness can arise from the representational substrate: a single continuous mapping is used irrespective of how many or which patches are masked. This does not eliminate dependence on training choices—the ablation on mask ratio shows a $f_\theta(x)\in[0,1]^3$5 optimum—but it reframes the sensitivity problem from mask engineering to function learning.

6. Extensions, scope, and relation to adjacent self-supervised paradigms

The paper presents MINR as a versatile self-supervised framework whose utility extends to various self-supervised learning applications. The future extensions listed are explicit: application to video through spatio-temporal coordinates mapping to RGB or motion vectors; generalization to other modalities such as audio waveforms and 3D point clouds via $f_\theta(x)\in[0,1]^3$6; and combination with contrastive or generative SSL tasks by extracting continuous-function embeddings (Lee et al., 30 Jul 2025).

These extensions remain prospective rather than experimentally established within the reported results. A plausible implication is that the essential abstraction in MINR is modality-agnostic function prediction by a masked-input hypernetwork. Under that reading, the image case is one instantiation of a broader coordinate-based self-supervised template.

In summary terms given by the paper, MINR replaces the discrete encoder-decoder of MAE with a hypernetwork-driven INR, learning a single continuous function per image from masked inputs. The reported consequences are superior in-domain and out-of-distribution reconstruction, substantial parameter reduction, and a naturally extensible framework for downstream vision and beyond.