---
title: 'ExpanDyNeRF: Monocular Dynamic Scene Reconstruction'
url: https://www.emergentmind.com/topics/expanded-dynamic-nerf-expandynerf
type: topic
---

# ExpanDyNeRF: Monocular Dynamic Scene Reconstruction

Expanded Dynamic NeRF (ExpanDyNeRF) is a monocular dynamic Neural Radiance Field (NeRF) framework designed for high-fidelity novel view synthesis of dynamic scenes under significant viewpoint deviations. The system advances the capability of monocular-only dynamic NeRFs by leveraging 3D Gaussian splatting priors and a pseudo–ground-truth generation strategy, achieving reliable scene reconstruction and photo-realistic renderings even when capturing scenes from a single, predominantly forward-facing camera with extreme viewpoint shifts up to and beyond ±45° [2512.14406].

## 1. Architecture and System Components

ExpanDyNeRF is predicated on a dual-branch NeRF backbone augmented with a novel pseudo–ground-truth loss from Gaussian splatting priors to address the failure modes of prior systems under large view deviations.

- **NeRF Backbone:** The architecture separates static and dynamic elements.
  - *Static Branch ($\mathcal{N}_b$):* Processes all frames $I_t$, learning a shared time-invariant density $\sigma_b(x)$ and color field $c_b(x, d)$.
  - *Dynamic Branch ($\mathcal{N}_f$):* Implements a 3-frame sliding-window NeRF; it takes $(x, d, t)$ and predicts time-varying density $\sigma_f(x,d,t)$ and appearance $c_f(x,d,t)$. Temporal coherence is enforced through an $L_2$ loss on adjacent densities.
- **Super-Resolution Module:** Low-resolution patches $\hat{Q}_k$ from volume rendering are upsampled with a pretrained SR model and compared with high-resolution ground truth patches $Q_k$.

The training process alternates or jointly optimizes backbone NeRF losses (on monocular views) with losses informed by the novel-view Gaussian priors.

## 2. Gaussian Splatting Priors and Alignment

Foremost in ExpanDyNeRF is the use of per-frame 3D Gaussian splatting priors, generated via FreeSplatter. Each foreground object is converted into a mesh of anisotropic Gaussians, yielding a volumetric proxy for appearance and geometry.

- **Novel View Dome Sampling:** For each frame, the system samples a dome of camera poses $P_{nv}^G(e, \phi)$ spanning $e \in \{0^\circ, 15^\circ, 30^\circ\}$ (elevation) and $\phi \in [-45^\circ, +45^\circ]$ (azimuth) in 5° steps, fixing the camera–object distance.
- **Pseudo–Ground-Truth Generation:** The Gaussian splat scene is rendered from each novel pose to produce color $c^*(r)$ and density $\sigma^*(r)$ for each ray.
- **Coordinate Alignment:** Each novel pose is mapped into the NeRF world through a rigid transformation $T = P_f^{NeRF} \cdot (P^G)^{-1}$, resulting in aligned rays for effective cross-domain supervision.

## 3. Loss Functions and Training Objective

The total objective is a composite of temporal, photometric, perceptual super-resolution, and novel-view supervision losses:

$$
L = L_{cont} + L_{rec} + L_{sr} + L_{nv}
$$

- **Temporal Continuity:** $L_{cont} = \mathbb{E}_t \|\sigma_f(x, d, t+1) - \sigma_f(x, d, t)\|_2^2$
- **Primary-View Reconstruction:** $L_{rec} = \sum_{t=1}^N \sum_{r \in R_t} \|I_t(r) - \hat{I}_t(r)\|_2^2$, using output from $\mathcal{N}_b \oplus \mathcal{N}_f$
- **Super-Resolution Perceptual Loss:** $L_{sr} = \sum_{k=1}^K \sum_{l \in layers} (\|\phi_l(\hat{Q}_k) - \phi_l(Q_k)\|_2^2) / |\phi_l|$, utilizing a VGG-19 feature extractor
- **Novel-View Pseudo–Ground-Truth Loss:**
  - $L_{nv} = L_c^{nv} + L_\sigma^{nv}$, with  
    $L_c^{nv} = \sum_{r \in R_{nv}} \|c_f(r) - c^*(r)\|_2^2$  
    $L_\sigma^{nv} = \sum_{r \in R_{nv}} \|\sigma_f(r) - \sigma^*(r)\|_2^2$  
  This loss is omitted for the initial $E_0$ epochs to avoid instability, then fully incorporated.

Hyperparameters are typically $\lambda_{cont}=1.0$, $\lambda_{rec}=1.0$, $\lambda_{sr}=0.5$, $\lambda_c^{nv}=\lambda_\sigma^{nv}=0.1$.

## 4. Synthetic Dynamic Multiview (SynDM) Dataset

ExpanDyNeRF’s novel-view supervision is enabled by SynDM, a synthetic multi-view dataset purpose-built using a GTA V-based rendering pipeline.

**Key properties of SynDM:**

| Attribute            | Details                                                      |
|----------------------|-------------------------------------------------------------|
| Multi-view sync      | 22 cameras (±45° azimuth at 5° steps, 3 at ±45° elevation)  |
| Temporal consistency | All cameras render in a single engine frame (0.2ms latency) |
| Scene variety        | Dynamic scenes (humans, vehicles, animals, unconstrained)   |
| Camera motion        | Main camera exhibits natural motion                         |
| Resolution/FOV       | 1920×1080 pixels, $90^\circ \times 59^\circ$ FOV            |
| Supervision splits   | Train: first 24 frames; Test: 12 novel cams (±5…±30°)       |

This dataset facilitates held-out, large-deviation ground truth for rigorous evaluation beyond small baseline shifts.

## 5. Empirical Performance and Ablations

ExpanDyNeRF demonstrates superior performance to state-of-the-art dynamic NeRF and splatting methods on both SynDM and real-world datasets, particularly for large azimuth deviations.

**Quantitative summary (SynDM, avg. over 9 scenes, ±30° test):**

| Method        | FID   | PSNR   | LPIPS  |
|---------------|-------|--------|--------|
| D3DGS         | 223.6 | 16.74  | 0.373  |
| DecNeRF       | 225.7 | 15.78  | 0.516  |
| RoDynRF       | 238.2 | 18.99  | 0.338  |
| DetRF         | 239.0 | 20.34  | 0.436  |
| **ExpanDyNeRF** | 142.7 | 20.86  | 0.209  |

On real-world DyNeRF [11] (most-deviated held-out cams):
- Scene “Coffee”: FID=132.4, PSNR=30.32, LPIPS=0.189 (2–6 point PSNR gain over best prior)
- Scene “Beef”: FID=135.8, PSNR=34.92, LPIPS=0.195

On NVIDIA [35] (small deviations): ExpanDyNeRF is a close second (PSNR ≈29–30, LPIPS ≈0.08–0.03), indicating robust generalization.

Qualitatively, baseline models exhibit severe artifacts (fragmented geometry, “cardboard” flattening, misplaced limbs, or collapse of thin structures) when tested at ±30° deviations. ExpanDyNeRF uniquely maintains sharp geometry, correct occlusion, and temporal color stability under these conditions.

## 6. Implementation Considerations

- Training is performed scene-wise for 300k iterations on dual NVIDIA A100 GPUs (approx. 15 h/scene).
- Gaussian splatting priors are generated once per frame with negligible overhead.
- The pseudo–ground-truth loss $L_{nv}$ is introduced after 50 warming epochs to allow backbone stabilization.

This sequence ensures the avoidance of gradient pathologies associated with noisy or unstable priors early in optimization.

## 7. Significance and Limitations

ExpanDyNeRF extends the applicability of dynamic NeRF frameworks to unconstrained, real-world video where only monocular, forward-facing camera input is available, and surpasses prior work in both metrics and qualitative stability under extreme viewpoint changes. The explicit use of 3D Gaussian splatting as a regularizing proxy, combined with the newly introduced SynDM dataset for large-deviation ground truth, establishes novel methodological and empirical standards in the field [2512.14406].

A plausible implication is that while ExpanDyNeRF robustly outperforms competitors on wide-baseline evaluation, its reliance on synthetic multitarget supervision may limit effectiveness where no proxy of this kind can be generated or where the alignment between proxy and real scene is poor. Nonetheless, its ability to preserve geometry and appearance under challenging view synthesis scenarios suggests its core techniques may inspire further advances in monocular dynamic scene reconstruction.

Source: https://www.emergentmind.com/topics/expanded-dynamic-nerf-expandynerf