Geometric Neural Process Fields

• Geometric Neural Process Fields are probabilistic neural field models that leverage hierarchical latent variables and geometric bases to infer distributions from sparse data.
• They integrate spatial inductive biases by compressing large context sets into structured, compact representations via learned, localized Gaussian bases.
• The hierarchical structure enables both global and local uncertainty estimation, enhancing reconstruction accuracy in images, 3D scenes, and signal regression.

Geometric Neural Process Fields (G-NPFs) are a probabilistic and hierarchical framework for generalizing neural fields, particularly in settings where limited observations are available and uncertainty quantification is crucial. Unlike traditional neural field models that specialize for a single signal via deterministic overfitting, G-NPFs explicitly parameterize entire distributions over neural fields, leverage structural geometric priors through learned geometric bases, and encode spatial information within a hierarchical latent variable architecture. This approach enables direct inference of implicit neural function distributions, improved generalization to unseen signals, and principled uncertainty estimation across data modalities including images, signals, and 3D neural radiance fields (Yin et al., 4 Feb 2025).

1. General Framework and Motivation

Geometric Neural Process Fields were developed in response to the poor generalization capacity of standard neural field models (NeFs, including NeRFs), which excel at signal reconstruction but lack adaptability in few-shot regimes. Standard practice requires training a unique overparameterized MLP per instance, a process that does not naturally account for uncertainty when data is scarce. G-NPFs reformulate neural field learning as a probabilistic inference problem: given a context set of sparse observations $\mathcal{C}$, the aim is to infer a distribution over neural fields $p(y_T | x_T, \mathcal{C})$ for a set of target coordinates $(x_T, y_T)$, allowing for flexible adaptation, proper epistemic uncertainty, and structural generalization.

The core predictive distribution is written as

$$p(y_T | x_T, \mathcal{C}) = \int p(y_T | x_T, z)\, p(z|\mathcal{C})\, dz$$

where $z$ denotes a latent variable encoding signal-level uncertainty, and $p(z|\mathcal{C})$ is inferred from context data via amortized inference (Yin et al., 4 Feb 2025). This Bayesian approach underpins all subsequent aspects of the model.

2. Geometric Bases for Structural Inductive Bias

To inject geometric inductive bias and encode spatial structure efficiently, G-NPF introduces a set of learned geometric bases. Instead of directly encoding the entire context $\mathcal{C}$ (often too large and redundant), the observed data are reduced to a compact set of $R$ geometric bases

$B_\mathcal{C} = \{b_r\}_{r=1}^R$

where each $b_r = (\mathcal{N}(\mu_r, \Sigma_r), \omega_r)$. Here, $\mathcal{N}(\mu_r, \Sigma_r)$ is a spatially localized Gaussian in the data domain (2D for images, 3D for NeRF scenes), and $\omega_r$ is a semantic feature embedding learned from the context. This design bottlenecks the representation, forcing the neural process to encode context with spatial locality and semantic attributes, thus imposing geometric consistency into the predicted field.

Aggregation of the bases for a target query $x_T$ is carried out via radial basis function (RBF) aggregation:

$$⟨x_T, B_\mathcal{C}⟩ = \text{MLP}\Bigg[\sum_{r=1}^R \exp\left(-\frac{1}{2}(x_T - \mu_r)^T \Sigma_r^{-1} (x_T - \mu_r)\right)\, \omega_r \Bigg]$$

The MLP post-aggregation introduces additional context-aware, non-linear transformations. The geometric bases, by compactly summarizing both spatial structure and semantic context, enable the learning of robust scene representations under few-shot supervision.

3. Hierarchical Latent Variable Model

G-NPF employs a hierarchical latent structure to reconcile the need for both global context and per-coordinate flexibility in neural field inference. The hierarchy is organized as follows:

• Global latent variable $z_g$: Captures global scene or signal properties by conditioning on all context information and geometric bases. This variable modulates the overall predicted function and allows the model to account for global uncertainty.
• Local latent variables $\{z_{\ell, m}\}$: For each target query (pixel, ray, or coordinate), a local latent variable is introduced, conditioned on the global latent and the coordinate/scene bases. This structure allows per-location adjustments and expresses fine-scale function variability.

The overall predictive distribution decomposes as:

$$p(y_T | x_T, B_\mathcal{C}) = \int p(z_g|x_T, B_\mathcal{C}) \prod_{m=1}^{|x_T|} \left[ \int p(y_{T,m}\,|\,z_g, z_{\ell,m}, x_{T,m}, B_\mathcal{C}) \,p(z_{\ell,m}|z_g, x_{T,m}, B_\mathcal{C}) dz_{\ell,m}\right] dz_g$$

In practice, inference is performed using amortized variational inference, with transformer encoders and MLPs estimating parameters (means and variances) for these Gaussians at both levels. This expressivity enables the model to represent both shared structure and local details in a data-efficient manner, and to propagate uncertainty throughout the field.

4. Uncertainty Quantification and Bayesian Modelling

Unlike prior NeF and NeRF variants, which provide only point estimates or overfit to the available data, G-NPFs treat function prediction as Bayesian inference, yielding explicit uncertainty estimates. Both the global and local latent variables contribute to the overall variance in predictions, enabling the model to maintain high uncertainty in underspecified regions (given few context points) and low uncertainty when context is dense.

Inference leverages the standard evidence lower bound (ELBO) from variational inference, regularizing the latent distributions to prior (standard Normal or learned scene-conditioned prior) distributions while maximizing the likelihood of observed data. This rigorous probabilistic foundation produces calibrated predictive distributions, suited for downstream tasks where uncertainty awareness is critical.

5. Generalization Experiments Across Modalities

The generalization ability of G-NPF is validated across a variety of data modalities:

• 2D image regression: On datasets such as CelebA and Imagenette, G-NPF achieves higher PSNR and improved reconstruction of both low- and high-frequency details compared to NeF ensemble baselines and meta-initialization methods (Yin et al., 4 Feb 2025).
• 3D novel view synthesis (NeRF): On ShapeNet and NeRF Synthetic scenes, G-NPF outperforms NeRF-VAE, PONP, and VNP in PSNR, SSIM, and LPIPS metrics under few-shot regimes (as low as 1–2 context images). The geometric bases and hierarchical inference enable realistic, detailed reconstructions when observations are sparse.
• 1D signal regression: On GP-sampled functions (RBF, Matern kernels), G-NPF shows higher log-likelihood and improved calibration compared to CNP, ANP, and VNP, demonstrating suitability well beyond visual domains.

The chosen architecture and priors yield generalization across both spatial and data modalities, enabling few-shot neural field learning in diverse settings.

6. Applications and Practical Implications

G-NPF advances the scope of implicit neural representations by equipping them with rapid adaptation and uncertainty quantification. Notable applications include:

• Computer vision and graphics: Enabling high-fidelity novel view synthesis and image inpainting from limited observations, facilitating design tasks where data is expensive or scarce.
• Robotics and autonomy: Providing uncertainty-calibrated scene/geometry prediction for navigation and perception in settings with incomplete sensory data.
• Medical imaging: Allowing shape and appearance interpolation from few data acquisitions, with uncertainty guides for diagnostic confidence.
• Scientific data modeling: Supporting implicit function modeling in cases where data is noisy or irregularly sampled.

The spatial and hierarchical structure of the model aligns with practical needs—such as the gradual integration of global context and precise, locality-aware inference at target coordinates.

7. Methodological Foundations and Future Directions

The methodological advances of G-NPF demonstrate that (a) reframing neural field adaptation as a probabilistic, hierarchical inference problem and (b) introducing geometric bases for structural regularization jointly enable strong generalization and robust uncertainty handling. The explicit Bayesian formulation makes the model extensible for integration with other neural process models, structured context encodings, or further geometric priors.

Potential future directions include: (i) extension to temporal or spatiotemporal neural fields with uncertainty-aware temporal dynamics, (ii) incorporation of richer symmetry or topology-specific bases, (iii) exploration of more complex context selectors or attention mechanisms to further improve data efficiency, and (iv) integration with foundation geometric neural operators (Quackenbush et al., 6 Mar 2025) to support direct geometric property inference and PDE solving.

In sum, Geometric Neural Process Fields represent a foundational probabilistic approach for adaptable, spatially-structured, and uncertainty-aware neural field modeling, applicable across a broad range of geometric and scientific domains (Yin et al., 4 Feb 2025).