Invertible Neural Radiance Fields

• InvNeRF models are formulations that support both forward image synthesis and inverse mapping to recover latent variables, camera pose, and scene parameters.
• Approaches range from zero-shot latent inversion in generative radiance fields to pose recovery via backpropagation and physically based inverse rendering techniques.
• These methods enhance applications in image editing, pose estimation, and surgical tracking by balancing practical inversion with varying degrees of architectural guarantees.

Invertible Neural Radiance Field (InvNeRF) denotes a family of radiance-field formulations in which a forward rendering process is paired with an inverse mapping, or is otherwise made “invertible” in practice, so that latent variables, camera pose, geometry, materials, or canonical coordinates can be recovered from observations. In ZIGNeRF, InvNeRF consists of a 3D-aware generative NeRF $G$ and an inverter $I$ such that the model supports both latent-to-image synthesis and image-to-latent inversion, with $I(G(z,\ldots)) \approx z$, enabling zero-shot inversion of out-of-domain images without model fine-tuning (Ko et al., 2023). Subsequent work uses the same or closely related idea for pose-invariant single-image inversion in style-based radiance fields (Li et al., 2022), joint recovery of shape, pose, and appearance from one image (Pavllo et al., 2022), photometric pose estimation by “inverting” a trained NeRF (Lin et al., 2022), physically based inverse rendering with a Neural Plenoptic Function (Wang et al., 2024), pose–NeRF joint optimization through an explicitly invertible neural warp (Chng et al., 2024), and bidirectional deformable-to-canonical mappings for surgical 2D and 3D tracking (Loza et al., 13 Aug 2025).

1. Terminological scope and problem formulation

The literature does not use InvNeRF in a single uniform sense. In some works it refers to latent inversion for a pretrained 3D-aware generator; in others it denotes camera-pose recovery, explicit invertible coordinate warps, or inverse rendering pipelines that recover scene parameters from images. A common thread is that a radiance-field model is not treated only as a forward renderer, but also as an object to be inverted or constrained by an inverse mapping.

Formulation	Inverse variables	Representative work
Generative radiance-field inversion	Latent code, sometimes disentangled into shape and appearance	ZIGNeRF (Ko et al., 2023)
Style-based NeRF inversion	View-invariant latent in $W$, refined in $W^+$	NeRF-3DE (Li et al., 2022)
Single-image radiance-field inversion	Shape, pose, and appearance	Bootstrapped radiance field inversion (Pavllo et al., 2022)
NeRF pose inversion	Camera extrinsics	Parallel inversion of NeRFs (Lin et al., 2022)
Inverse rendering with radiance fields	Geometry, materials, lighting	Neural Plenoptic Function method (Wang et al., 2024)
Explicitly invertible pose warp	World–camera warp with architectural invertibility	Invertible Neural Warp for NeRF (Chng et al., 2024)
Deformable canonical mapping	Bidirectional observation–canonical mapping for tracking	Surg-InvNeRF (Loza et al., 13 Aug 2025)

A central distinction in this literature is between practical invertibility and strict invertibility. Encoder-based inversion methods approximate an inverse map from image space to a generator manifold, but they are not bijective. NeRF-3DE states this explicitly: encoder-based inversion “is not bijective,” whereas strictly invertible architectures such as normalizing flows would require a truly invertible radiance-field parameterization and tractable Jacobians (Li et al., 2022). This suggests that the term invertible is often operational rather than formal: the model is made invertible enough for reconstruction, editing, pose recovery, or tracking, even when exact bijection is absent.

2. Latent inversion in generative radiance fields

In ZIGNeRF, InvNeRF is defined by two jointly designed components: a 3D-aware generative NeRF $G$ and an inverter $I$. The generator follows a compositional GIRAFFE-style neural feature field. A 3D point $x \in \mathbb{R}^3$, view direction $d \in S^2$, and latent code $z$ are mapped to density $I$0 and feature $I$1. Object-level transforms are parameterized by $I$2, with $I$3 and $I$4, and

$I$5

The composed feature field is volume-rendered to a 2D feature map $I$6, and a neural renderer $I$7 maps $I$8 to an image: $I$9 The inverse process maps an input image back to the generator manifold,

$I(G(z,\ldots)) \approx z$0

and reconstructs through the frozen generator: $I(G(z,\ldots)) \approx z$1 The inverter outputs four latent vectors, $I(G(z,\ldots)) \approx z$2, corresponding to foreground-object and background shape and appearance (Ko et al., 2023).

The ZIGNeRF inverter is trained only on synthetic samples rendered by the pretrained generator and discriminator, which are then frozen. Its objective combines adversarial realism, latent reconstruction, image reconstruction, and perceptual similarity: $I(G(z,\ldots)) \approx z$3 This formulation is explicitly zero-shot: the inverter is trained once on generator-produced images and then applied to real or out-of-domain inputs without per-image optimization or fine-tuning. The model does not estimate pose during inversion; camera pose $I(G(z,\ldots)) \approx z$4 is controlled at render time, and 3D operations are realized by varying $I(G(z,\ldots)) \approx z$5 and $I(G(z,\ldots)) \approx z$6 while keeping the inverted latent fixed (Ko et al., 2023).

NeRF-3DE addresses the same inversion problem for style-based radiance fields such as StyleNeRF and EG3D, but emphasizes view-invariance of the inferred latent. Its first-stage encoder predicts a 512-dimensional latent $I(G(z,\ldots)) \approx z$7, and its second-stage encoder predicts per-layer residuals $I(G(z,\ldots)) \approx z$8, yielding

$I(G(z,\ldots)) \approx z$9

The base stage is trained so that multiple views of the same synthetic identity map to the same latent through triplet supervision and latent $W$0 recovery, while image-space reconstruction, LPIPS, and ArcFace-based identity loss preserve appearance (Li et al., 2022). The stated rationale is that inversion should first locate “the closest latent code lying on the latent manifold,” after which restrained $W$1 refinement can recover details without sacrificing 3D consistency.

Both models treat latent inversion as a 3D-aware problem rather than a conventional 2D GAN-inversion problem. In ZIGNeRF this appears as object-background disentanglement and explicit object transforms; in NeRF-3DE it appears as pose-invariant encoding across synthetic multi-view tuples. A plausible implication is that the inverse map is most useful when it respects the factorization already imposed by the forward radiance-field generator.

3. Pose, shape, and camera recovery by NeRF inversion

A second use of InvNeRF centers on recovering camera pose, or jointly recovering pose with scene variables, by optimizing against a fixed or pretrained radiance field. In “Parallel Inversion of Neural Radiance Fields for Robust Pose Estimation,” the inverse problem is

$W$2

where $W$3 is rendered by the NeRF under candidate extrinsics and known intrinsics. The method departs from coupled $W$4 optimization by using $W$5, maintaining separate momentum updates for rotation and translation. Rotation updates are induced by a torque-like quantity

$W$6

while translation follows the sum of pointwise rendering gradients (Lin et al., 2022).

The same work introduces parallel Monte Carlo search over $W$7 pose hypotheses, a free-exploration phase, and repeated resampling around the best survivors. Instant-NGP supplies the fast forward and backward passes that make this feasible. The paper also evaluates several robust pixel losses and recommends MAPE as a robust default, while noting that Relative $W$8 performs best under synthetic noise and $W$9 is best on LLFF (Lin et al., 2022). Here inversion means recovering camera extrinsics from a single image by backpropagating through volume rendering.

Bootstrapped radiance field inversion extends this idea to single-image recovery of shape, pose, and appearance. The method first trains an unconditional EG3D-like generator with an SDF parameterization,

$W^+$0

regularized by the Eikonal loss,

$W^+$1

A synthetic-data encoder then predicts a latent code $W^+$2, a segmentation mask, and a canonical NOCS-style map. Pose initialization is obtained by SQPnP from the predicted canonical map, and a short hybrid optimization refines both latent and pose: $W^+$3 The method reports practical inversion in as few as 10 gradient steps and is explicitly designed for settings without multi-view training data or ground-truth poses (Pavllo et al., 2022).

This branch of the literature makes clear that InvNeRF need not mean latent inversion alone. It may instead mean inversion of the rendering process with respect to camera or scene parameters, provided gradients can be propagated through the radiance-field renderer and the optimization is sufficiently conditioned.

4. Explicit invertibility, canonical mappings, and deformation

A stricter interpretation appears in “Invertible Neural Warp for NeRF.” Rather than directly optimizing per-image $W^+$4 variables, the method replaces explicit camera pose with a learnable rigid warp $W^+$5 implemented by an invertible neural network. Camera-frame coordinates and the camera center are warped to world-frame coordinates,

$W^+$6

after which the NeRF renders along the transformed ray. The final objective is

$W^+$7

Invertibility is guaranteed by the INN architecture itself, not by a penalty, and a geometry-informed rigidity prior is enforced through a best-fit rigid transform $W^+$8 obtained from current correspondences (Chng et al., 2024).

This work explicitly argues that invertibility is critical because camera-to-world warps must remain bijective; naive MLP warps can fold or collapse. The paper characterizes the INN as predicting homeomorphisms, which are more flexible than rigid $W^+$9 trajectories during optimization, yet can be softly anchored toward rigid motion through $G$0. This is one of the clearest examples in which InvNeRF denotes architectural invertibility rather than merely an encoder that approximates an inverse.

Surg-InvNeRF adopts a related but distinct notion for deformable scenes. It defines a bidirectional mapping between a time-varying observation space and a static canonical space: $G$1 Radiance and density are defined only in the deformed observation space, while canonical space serves consistency and tracking. The method combines photometric rendering, reprojection supervision from external 2D correspondences, triangulation-guided density supervision, and canonical regularization: $G$2 The canonical constraints include cross-consistency,

$G$3

and a sphere constraint,

$G$4

The deformation field is implemented with CaDeX/CaDeX++ blocks, while the appearance and density fields are represented with Multi-scale HexPlanes for speed (Loza et al., 13 Aug 2025).

Unlike the INN warp paper, Surg-InvNeRF does not impose explicit Jacobian-related constraints or log-determinant losses. Bijectivity is promoted through cycle-consistency-like structure, canonical bounds, and multi-space supervision. This suggests a spectrum of “invertibility” claims: exact architectural invertibility at one end, and empirically stabilized bidirectional deformation mappings at the other.

5. Inverse rendering and physically based generalizations

A further extension of the InvNeRF idea appears in inverse rendering, where the inverse problem concerns geometry, BRDF parameters, and lighting rather than latent codes or camera pose. The Neural Plenoptic Function method introduces a 5D incident-radiance field

$G$5

and uses it within the rendering equation

$G$6

Its pipeline is two-stage: reconstruct geometry and an environmental radiance field first, then estimate spatially varying materials $G$7 using ray tracing, a microfacet BRDF, and material-aware cone sampling (Wang et al., 2024).

The method is motivated by the claim that conventional NeRF-based inverse rendering usually assumes a 2D environment map, which cannot model finite-distance lighting, local occlusion, or interreflection for glossy objects. The proposed Neural Plenoptic Function is combined with a Mip-NeRF environmental field so that pre-filtered incident radiance can be queried over cones aligned with BRDF lobes. For glossy materials, the paper reports that cone sampling can achieve comparable quality with approximately 8 diffuse and 4 specular cones, whereas typical Monte Carlo sampling may require approximately 256 diffuse and 128 specular rays (Wang et al., 2024).

The paper is explicit that this is not a closed-form mathematical inverse of NeRF or of the rendering equation. Rather, it is a practical inverse-rendering pipeline that makes a learned radiance field “invertible” in the sense of recoverable geometry, lighting, and materials. In this usage, InvNeRF denotes structured inversion of the rendering process into exportable, conventional rendering assets.

6. Empirical behavior, limitations, and common points of confusion

Representative quantitative results illustrate how heterogeneous the InvNeRF literature is.

Work	Reported result	Interpretation
ZIGNeRF	Cats FID at $G$8: 12.31 vs 24.01 for GIRAFFE; CelebA(HQ) conditional FID at $G$9: 14.77; inverter ablation on CelebA(HQ) $I$0: 80.08 $I$1 17.82 $I$2 15.53 $I$3 14.77 (Ko et al., 2023)	Zero-shot latent inversion and rendering improve over the baseline generator and require all inverter loss terms
NeRF-3DE	Same-view CelebA-HQ reconstruction: $I$4, LPIPS $I$5, ID $I$6, $I$7 s; novel-view ArcFace cosine average $I$8 (Li et al., 2022)	Two-stage pose-invariant inversion improves 3D-consistent novel-view identity retention
Bootstrapped inversion	Pascal3D+ Cars: IoU up to 0.920 and FID $I$9; SRN Cars: PSNR 19.55, SSIM 0.864, FID 11.37 (Pavllo et al., 2022)	Hybrid inversion with synthetic bootstrapping is practical without multi-view training
Parallel NeRF inversion	15–20 s per image inversion on RTX 3090; LLFF success with multiple hypotheses reaches 1.00 for both rotation $x \in \mathbb{R}^3$0 and translation $x \in \mathbb{R}^3$1 (Lin et al., 2022)	Fast photometric pose inversion benefits from parallel hypothesis search and robust losses
Invertible Neural Warp	LLFF average pose error: rotation $x \in \mathbb{R}^3$2, translation $x \in \mathbb{R}^3$3; DTU mean Chamfer distance 4.89 (Chng et al., 2024)	Explicit INN-based invertibility improves joint pose–NeRF optimization convergence
Neural Plenoptic Function	Average Chamfer Distance: NeRO 0.00322 vs Ours 0.00164; material MSE: 0.003 / 0.020 / 0.036 for roughness / metallic / albedo (Wang et al., 2024)	Inverse rendering with finite-distance lighting improves glossy-object reconstruction
Surg-InvNeRF	STIR subset 2D EPE: 13.925 px vs Omnimotion 31.793 px and FastOmniTrack 26.863 px; 3D tracking EPE $x \in \mathbb{R}^3$4 mm (Loza et al., 13 Aug 2025)	Bidirectional canonical mapping enables long-term 2D and 3D tracking in surgical scenes

Several limitations recur across the literature. ZIGNeRF does not estimate pose during inversion, so the viewpoint of a reconstructed real image may not match the input unless $x \in \mathbb{R}^3$5 is set accordingly; extreme out-of-domain inputs can also degrade disentanglement or texture fidelity. NeRF-3DE emphasizes single-view ambiguity, domain gaps, camera-estimation errors, and the limited yaw range to which StyleNeRF generalizes reliably. Bootstrapped radiance field inversion reports local-minimum failures under pose ambiguity and difficulty with thin structures or highly reflective surfaces on small datasets. Pose-only inversion methods assume known intrinsics and are sensitive to weak texture or large initial misalignment. The Neural Plenoptic Function method notes that material estimation depends on accurate geometry and that transparent or translucent materials are out of scope. Surg-InvNeRF relies on accurate short-term correspondences and remains brittle under smoke, bleeding, or severe deformation.

The most common conceptual confusion is to assume that every method labeled InvNeRF is mathematically invertible in the same sense. The papers do not support that interpretation. In ZIGNeRF and NeRF-3DE, invertibility is learned as an image-to-latent mapping on or near a generator manifold. In bootstrapped inversion and photometric pose inversion, it is an optimization problem through a renderer. In inverse rendering, it is recovery of explicit scene parameters from radiance observations. Only the invertible-warp formulation makes invertibility an architectural guarantee, and even there rigidity is imposed softly rather than by constraining the network to $x \in \mathbb{R}^3$6. A plausible implication is that InvNeRF is best understood as a research direction—making radiance fields recoverable from observations under increasingly structured inverse maps—rather than as a single model class.