DeblurGS: 3D Scene Reconstruction & Deblurring

• DeblurGS is an umbrella framework of physically grounded methods that uses 3D Gaussian Splatting to reconstruct scenes and deblur images.
• It integrates explicit camera motion and exposure modeling with differentiable rendering to recover high-fidelity, temporally coherent scenes from severe blur.
• The approach achieves state-of-the-art performance in novel-view synthesis and video deblurring, outperforming traditional methods on metrics like PSNR and SSIM.

DeblurGS is the umbrella term for a series of physically grounded, differentiable frameworks for 3D scene reconstruction and image deblurring from motion-blurred inputs, leveraging 3D Gaussian Splatting (3DGS) as the core scene representation. These methods achieve state-of-the-art performance in photorealistic, temporally coherent deblurring and novel-view synthesis from both static and dynamic, multi-view and monocular, and even extremely blurred visual data, through explicit modeling of camera and object motion during exposure and sophisticated optimization strategies (Oh et al., 2024, Zhao et al., 14 Oct 2025, Matta et al., 20 Aug 2025, Niu et al., 2024, Wu et al., 2024, Bui et al., 21 Apr 2025).

1. Mathematical Framework and Scene Representation

DeblurGS builds on the explicit 3DGS model, representing a scene as a set of anisotropic 3D Gaussians $G = \{(\mu_i, \Sigma_i, o_i, c_i)\}_{i=1}^N$, where:

• $\mu_i \in \mathbb{R}^3$: Gaussian center,
• $\Sigma_i \in \mathbb{R}^{3 \times 3}$: covariance,
• $o_i$: opacity,
• $c_i$: color (typically view-dependent, parameterized via SH bases).

Each Gaussian undergoes SE(3) transformation for camera pose, and is projected with a Jacobian-derived covariance into the image domain. Rendering follows per-pixel alpha-compositing in depth order.

A core advance in DeblurGS is its explicit, physically inspired model of camera-induced motion blur: $$B(u) = \frac{1}{N_\text{sub}} \sum_{j=1}^{N_\text{sub}} R(G, P_j)[u]$$ where $B(u)$ is the observed blurred pixel, $P_j$ denotes the interpolated camera pose at sub-timestamp $j$ within the exposure, and $R$ is the rasterization operator for the set of Gaussians (Oh et al., 2024, Matta et al., 20 Aug 2025).

This physically justified integration is the basis for the joint optimization of both Gaussian parameters and camera (or object) motion, enabling the explicit recovery of blur-free scenes even from low-quality, blur-dominated inputs.

2. Camera and Trajectory Modeling

A central challenge addressed by DeblurGS is the accurate estimation and exploitation of camera motion trajectories from blurred views where structure-from-motion (SfM) can provide only noisy or incomplete pose initialization (Oh et al., 2024, Matta et al., 20 Aug 2025).

Common techniques for trajectory and exposure modeling include:

• Continuous trajectory parameterization: Camera motion during the exposure interval $\mu_i \in \mathbb{R}^3$0 is represented as a high-order Bézier curve in se(3), $\mu_i \in \mathbb{R}^3$1, or as interpolations between endpoint poses. The sub-frame alignment parameters $\mu_i \in \mathbb{R}^3$2 allow extra flexibility to adapt to real blur kernel shape (Oh et al., 2024, Zhao et al., 14 Oct 2025).
• Exposure estimation: In frameworks such as Deblur4DGS, a single scalar exposure per frame is estimated rather than a dense trajectory, greatly reducing the parameter space and promoting consistency (Wu et al., 2024).
• Sequence-level strategies: Bi-stage approaches (e.g., BSGS) refine endpoint camera poses in a first phase, and introduce a global rigid correction in a second phase to capture remaining misalignments, employing special gradient aggregation strategies to stabilize optimization (Zhao et al., 14 Oct 2025).
• Event data integration: In situations of extreme blur, event camera streams allow generation of pseudo-sharp latent frames (via EDI) for improved pose initialization and subsequent optimization (Weng et al., 2024, Matta et al., 20 Aug 2025).

3. Optimization and Loss Functions

DeblurGS-type methods jointly optimize over Gaussian primitive parameters, camera/object trajectory splines or endpoints, and potentially exposure times. The primary losses include:

• Photometric (reconstruction) loss: Typically an $\mu_i \in \mathbb{R}^3$3 distance between observed blurred images and rendered machine-blurred predictions, potentially augmented with a D-SSIM structural similarity term:

$\mu_i \in \mathbb{R}^3$4

(Oh et al., 2024, Zhao et al., 14 Oct 2025)

• Temporal smoothness: Penalizes abrupt changes between adjacent sub-frame renders to enforce physically plausible trajectory evolution (Oh et al., 2024).
• Exposure and regularization losses: Prevent collapse to trivial solutions (e.g., zero exposure/blur), and enforce consistency across time (multi-frame, multi-resolution).
• Motion decomposition and temporal consistency: Particularly for dynamic scenes, unsupervised regularizers (entropy, sparsity) enforce static/dynamic separation; cycle-consistency terms on reprojected flows and latent frames propagate detail and maintain correspondence across time (Bui et al., 21 Apr 2025, Wu et al., 2024).

Gradient flow is maintained through the entire differentiable pipeline, including the SE(3) trajectory, Gaussian properties, and the rendering/integration operation.

4. Implementation Strategies and Architectural Variations

A variety of initialization and densification strategies are in common use:

• SfM and COLMAP initialization: Used for initial pose and point cloud estimates, but fragile under heavy blur (Oh et al., 2024, Matta et al., 20 Aug 2025). Event-based or learned SfM (VGGSfM) can provide robust alternatives (Weng et al., 2024, Matta et al., 20 Aug 2025).
• Gaussian densification/pruning heuristics: Early versions use thresholds on gradient norm to split or prune Gaussians. Densification Annealing delays splitting until camera pose parameters converge, avoiding the placement of spurious, non-explanatory Gaussians (Oh et al., 2024, Zhao et al., 14 Oct 2025).
• Space-time adaptive densification in bi-stage methods: The splitting threshold dynamically adapts in both space (depth from the camera) and time (training phase), preventing noisy Gaussian growth in initial blurry regions and allowing high-frequency detail recovery once poses are stable (Zhao et al., 14 Oct 2025).
• Event-driven frameworks: E.g., EaDeblur-GS integrates event-based pseudo-sharp images and an Adaptive Deviation Estimator (ADE) network to guide per-Gaussian corrections, leading to high stability and extremely fast inference, at the cost of requiring event camera hardware (Weng et al., 2024).

5. Applications and Empirical Performance

DeblurGS-family approaches demonstrate robust real-world and synthetic performance across a range of downstream tasks:

• Novel-view synthesis: High-fidelity, real-time rendering (up to 30 fps on 1080p in Deblur4DGS) of previously unseen views from blurred video (Wu et al., 2024, Oh et al., 2024).
• Video deblurring: Outperforms prior methods, including NeRF- and GAN-based baselines, across PSNR, SSIM, and LPIPS metrics. For example, DeblurGS and BSGS report gains of several dB PSNR and 0.1-0.16 in SSIM over BAD-GS and DyBluRF on established benchmarks (Oh et al., 2024, Zhao et al., 14 Oct 2025).
• Dynamic (4D) scene reconstruction: Methods such as Deblur4DGS and MoBGS explicitly address dynamic content and jointly estimate temporally coherent geometry and appearance (Wu et al., 2024, Bui et al., 21 Apr 2025).
• Avatar modeling from blurred multispectral video: Incorporates explicit dynamic blur formation and joint bundle adjustment for avatar recovery from multi-view blurry footage, superior to two-stage pipelines (Niu et al., 2024).
• Event-driven deblurring and reconstruction: Allows robust operation on severe and even extreme blur, exceeding prior 3DGS and NeRF methods both in quality and inference speed (Weng et al., 2024, Matta et al., 20 Aug 2025).

Empirical results highlight consistently high performance in image-space and perceptual metrics, despite degraded or noisy input poses.

Method	PSNR (dB)	SSIM	LPIPS	Notes
DeblurGS (MoBGS)	28.7	0.945	0.051	State-of-the-art dynamic NVS (Bui et al., 21 Apr 2025)
Deblur4DGS	28.9	0.949	0.060	4D dynamic scenes (Wu et al., 2024)
BSGS	29.1–32.1	0.82–0.91	0.08–0.14	On real/synthetic motion blur (Zhao et al., 14 Oct 2025)
EaDeblur-GS	30.08	0.937	—	Real-time, event-aided (Weng et al., 2024)
DeblurGS [original]	26.3–31.8	0.806–0.890	0.086–0.172	Moderate-to-noisy pose settings (Oh et al., 2024)

A plausible implication is that explicit and physically motivated integration of camera/object motion in the blur model, coupled with end-to-end differentiable optimization, is critical for state-of-the-art deblurring—especially where conventional SfM and radiance field methods fail.

6. Limitations and Open Challenges

Despite their performance, DeblurGS-type frameworks have several notable constraints:

• Pose initialization fragility: Performance degrades if initial SfM fails due to excessive blur; advanced initialization strategies (deep SfM, event cues) are under active research (Matta et al., 20 Aug 2025).
• Computational cost: Joint optimization, especially of high-order camera splines and large Gaussian sets, can be slow ($\mu_i \in \mathbb{R}^3$5k iters, hours per sequence) (Oh et al., 2024).
• Handling dynamic, non-rigid and rolling-shutter scenes: Current methods primarily support rigid or smoothly deforming objects; generalization to arbitrary non-uniform motion and sensor characteristics remains open (Wu et al., 2024, Niu et al., 2024).
• Foreground–background mixing and segmentation: Some methods require mask pre-computation and are limited to static or segmented backgrounds (Niu et al., 2024).
• Extreme blur and failure cases: Weak texture, high reflectivity, or no detectable correspondences still break standard pipelines.

7. Extensions and Future Research Directions

Key avenues for future improvement have been identified:

• Integration of sensor modalities: Combining visual data with inertial/gyroscopic information or event cameras for robust pose under extreme motion (Weng et al., 2024, Matta et al., 20 Aug 2025).
• Deformable and dynamic scene blur: Integration of optical-flow priors, multi-scale exposure modeling, and per-Gaussian deformation fields to generalize beyond current motion models (Wu et al., 2024).
• End-to-end learning: Jointly train all parameters—geometry, appearance, pose, exposure—in a unified MLP or transformer architecture for robustness and scalability (Wu et al., 2024).
• Adaptive densification/splitting: Further development of sampling-based or probabilistic densification, subsuming hand-crafted heuristics (Matta et al., 20 Aug 2025).
• Rolling-shutter and varying exposure: Extend frameworks to support non-uniform, sensor-dependent blur, eg via rolling-exposure modeling (Wu et al., 2024).

A plausible implication is that advances in learning-based initialization, sensor fusion, and unified pipeline optimization will enable DeblurGS methods to extend to even more challenging domains, such as in-the-wild dynamic scenes, mobile/lidar fusion, and streaming video deblurring.

---

Key references: (Oh et al., 2024, Wu et al., 2024, Niu et al., 2024, Zhao et al., 14 Oct 2025, Bui et al., 21 Apr 2025, Matta et al., 20 Aug 2025, Weng et al., 2024).