DeblurNVS: Latent Diffusion for Novel Views

• The paper introduces DeblurNVS, a framework that restores intermediate geometric representations from sparse motion-blurred images using latent diffusion to enable sharp novel view synthesis.
• It employs a two-stage process where blurred inputs are first encoded with a pretrained geometric encoder and LoRA adapters, and then refined via context and target-view denoisers conditioned on camera parameters.
• Empirical results demonstrate competitive PSNR and SSIM with improved perceptual quality (LPIPS, FID) and fast runtime compared to scene-specific methods.

DeblurNVS is a framework for novel view synthesis from sparse motion-blurred images that aims to produce a sharp target-view image from calibrated blurred context views without per-scene optimization. The method is formulated around the observation that motion blur weakens local image structure and cross-view geometric cues, which are normally assumed by NeRF, 3D Gaussian Splatting, and generative view synthesis pipelines. DeblurNVS addresses this by restoring intermediate geometric representations before target-view synthesis, then combining those restored representations with target camera information to reconstruct a sharp RGB novel view. The framework is presented in "DeblurNVS: Geometric Latent Diffusion for Novel View Synthesis from Sparse Motion-Blurred Images" (Shi et al., 31 May 2026).

1. Problem setting and image formation

Novel view synthesis (NVS) is posed as the task of predicting a target-view image from a sparse set of observed views. In DeblurNVS, the observed inputs are not clean images but real-world motion-blurred context images

$$\mathcal I_c^{\rm blur}=\{\mathbf I_i^{\rm blur}\}_{i=1}^K$$

with calibrated camera poses, and the objective is to synthesize a sharp target-view image

$\mathbf I^{\star}$

from a novel camera pose, without any per-scene optimization (Shi et al., 31 May 2026).

The motion-blur model follows finite-exposure image formation. A blurred measurement is defined as the time integral of the instantaneous sharp radiance: $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$ Here, $\mathbf I^{\rm sharp}(t)$ denotes the latent sharp image at time $t$, $T$ is the exposure duration, and the practical approximation is obtained through frame interpolation and temporal averaging. This formulation makes explicit that blur is not treated as an additive corruption but as an exposure-time aggregation of latent sharp observations.

The paper situates this setting against two limitations of prior practice. First, most NVS systems rely on clean observations, where image structures and cross-view correspondences remain intact. Second, blur-aware methods that model image formation typically depend on costly per-scene optimization, which restricts efficient and generalizable sparse-view synthesis. DeblurNVS is introduced specifically to remove that dependence while preserving multi-view reasoning under blur.

2. Geometry-aware latent restoration

A central design choice in DeblurNVS is to restore a geometry-aware context latent from each blurred input before performing view synthesis. The method uses a pretrained geometric encoder $E_{\rm DA3}$, identified as Depth Anything 3, and augments it with lightweight LoRA adapters. Each blurred image is encoded as

$$\widetilde{\mathbf z}_i =E_{\rm DA3}^{\rm LoRA}\bigl(\mathbf I_i^{\rm blur}\bigr), \qquad \widetilde{\mathcal Z}_c=\{\widetilde{\mathbf z}_i\}_{i=1}^K.$$

The restored context geometry is learned through a latent diffusion process. A frozen teacher encoder produces sharp reference latents $$\mathcal Z_c^{\rm sharp}=\{\mathbf z_i^{\rm sharp}\}$$. A U-Net denoiser $\epsilon_\theta^{\rm ctx}$ is then trained to predict noise from a noised version of the sharp latent, conditioned on the blurred latent set: $\mathbf I^{\star}$0

$\mathbf I^{\star}$1

Minimizing $\mathbf I^{\star}$2 jointly updates the LoRA adapters and the context denoiser while keeping the teacher encoder frozen (Shi et al., 31 May 2026). Because $\mathbf I^{\star}$3 processes all $\mathbf I^{\star}$4 latents jointly and operates in the DA3 feature space, which the paper describes as encoding underlying depth-geometry priors, the restored latents $\mathbf I^{\star}$5 are reported to inherit strong multi-view geometric alignment. The paper further states that no additional 2D feature-alignment loss is required in practice; the consistency emerges from the shared latent-space diffusion.

This stage is significant because it separates the restoration of structure and correspondence cues from the subsequent generation of the target view. The paper characterizes this as reconstructing the intermediate geometric representations needed for multi-view reasoning from blurred observations.

3. Target-view synthesis by geometric latent diffusion

After recovering the restored context latents $\mathbf I^{\star}$6, DeblurNVS learns to generate the target-view latent rather than directly predicting RGB. The target sharp image is first mapped into the same geometric latent space using the teacher encoder: $\mathbf I^{\star}$7 A forward noising process constructs

$\mathbf I^{\star}$8

A second denoiser, $\mathbf I^{\star}$9, is trained to predict the noise at each diffusion step while conditioning on the restored context latents and the full set of camera embeddings $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$0: $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$1

The paper interprets this as geometry-aware diffusion: the restored context latents provide structural information such as depth and parallax, and the camera embeddings provide the novel-view camera parameters, so the reverse process converges to a target latent $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$2 that is both sharp and consistent with multi-view geometry (Shi et al., 31 May 2026).

This formulation distinguishes DeblurNVS from approaches that synthesize directly from degraded RGB or depend on scene-specific optimization. The method instead uses a two-stage latent construction in which deblurring and view synthesis are coupled through a shared geometric representation.

4. RGB reconstruction and supervised objectives

Once the target latent $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$3 has been sampled, DeblurNVS reconstructs the final image with a lightweight decoder $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$4: $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$5

The decoder is supervised using a weighted combination of image, perceptual, and adversarial losses: $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$6 with $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$7, $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$8, and $$\mathbf I^{\rm blur} = \frac{1}{T}\int_{t_0}^{t_0+T} \mathbf I^{\rm sharp}(t)\,\mathrm dt \;\approx\; \frac{1}{N}\sum_{i=1}^N \mathbf I_i^{\rm sharp}, \qquad N\sim\mathcal U\{5,7,9,11\}.$$9.

The loss design reflects the paper’s emphasis on both perceptual sharpness and structural fidelity. The $\mathbf I^{\rm sharp}(t)$0 term anchors reconstruction to the sharp target image, LPIPS contributes perceptual supervision, and the GAN term is used in the final RGB stage. The resulting pipeline therefore proceeds from blurred image encoding, to geometric latent restoration, to camera-conditioned target latent diffusion, and finally to RGB decoding (Shi et al., 31 May 2026).

A common misunderstanding would be to reduce the method to a sequential “deblur then render” pipeline. The paper’s own description is more specific: DeblurNVS restores intermediate geometric representations and then synthesizes the target-view representation in the same latent regime. This suggests that the deblurring operation is organized around multi-view geometry rather than generic single-image restoration.

5. Dataset construction and training configuration

Large-scale training is enabled by a dedicated motion-blurred NVS dataset derived from DL3DV-10K. Starting from DL3DV-10K, described as containing approximately 10,000 scenes with approximately 500 frames each, the training data are generated by interpolating each view sequence $\mathbf I^{\rm sharp}(t)$1 in time and then averaging $\mathbf I^{\rm sharp}(t)$2 frames, where $\mathbf I^{\rm sharp}(t)$3, to synthesize realistic motion blur. The resulting dataset contains approximately 5 million sharp–blur pairs for supervision (Shi et al., 31 May 2026).

The training resolution is $\mathbf I^{\rm sharp}(t)$4. Context diffusion uses 8 denoising steps. Target diffusion uses 12 denoising steps with an Euler sampler. Batch size and learning rate are stated to follow standard DA3/GLD defaults. Overall training takes 2–3 days on a single 48 GB A100/4090 GPU. Data augmentation consists of random horizontal flips, color jitter, and slight camera-pose perturbations.

The dataset construction is important because the abstract explicitly ties the framework’s scalability to interpolation-based finite-exposure blur synthesis. The synthetic blur generation directly mirrors the image-formation model used in the formulation section, giving the training procedure a consistent exposure-based interpretation rather than a purely heuristic corruption model.

6. Empirical results and method positioning

The reported experiments use 3 input views for the key quantitative comparisons. On synthetic motion-blur benchmarks, DeblurNVS is presented as the only generalizable method trained end-to-end for blurred inputs, whereas the scene-specific baselines 3DGS and BAGS require minutes of per-scene optimization at test time (Shi et al., 31 May 2026).

Synthetic DL3DV-Bench, blurred inputs

Method	Results
3DGS (scene-opt.)	PSNR 14.77; SSIM 0.527; LPIPS 0.422; FID 233.4; time ≈32 min
BAGS (scene-opt.)	PSNR 15.18; SSIM 0.539; LPIPS 0.414; FID 221.8; time ≈12 min
DA3 (gen.)	PSNR 10.95; SSIM 0.357; LPIPS 0.581; FID 275.5; time 0.03 s
GLD (gen.)	PSNR 11.46; SSIM 0.385; LPIPS 0.503; FID 150.9; time 9.8 s
DeblurNVS	PSNR 15.55; SSIM 0.441; LPIPS 0.367; FID 101.4; time 0.60 s

Real DeblurNeRF dataset, 3 inputs

Method	Results
BAGS (scene-opt.)	PSNR 18.81; SSIM 0.527; LPIPS 0.385; FID 151.1; time ≈30 min
DA3 (gen.)	PSNR 13.61; SSIM 0.356; LPIPS 0.608; FID 306.2; time 0.03 s
GLD (gen.)	PSNR 12.87; SSIM 0.329; LPIPS 0.570; FID 166.5; time 9.8 s
DeblurNVS	PSNR 17.13; SSIM 0.433; LPIPS 0.335; FID 79.6; time 0.60 s

Across all benchmarks, the paper states that DeblurNVS delivers the best perceptual metrics, specifically LPIPS and FID, together with very competitive PSNR and SSIM, while running in under 1 second per novel view and avoiding per-scene tuning. Qualitatively, it is reported to produce sharper edges, more faithful geometry with fewer “floating” artifacts, and stable textures under strong motion blur; by contrast, scene-specific 3DGS and BAGS as well as clean-input generalizable methods DA3 and GLD are described as struggling when correspondences are degraded by blur.

The method is summarized in the paper as the first generalizable, geometry-aware latent-diffusion framework that simultaneously disentangles multi-view deblurring from novel-view synthesis in a shared latent space, reconstructs reliable multi-view geometry from sparse blurred observations, and synthesizes high-fidelity sharp novel views without per-scene optimization (Shi et al., 31 May 2026). A plausible implication is that DeblurNVS occupies a hybrid position between blur-aware reconstruction and generalizable view synthesis: it retains explicit geometric conditioning while replacing scene-specific test-time optimization with learned latent diffusion.