Plug-and-Play PDE Optimization for 3D Gaussian Splatting: Toward High-Quality Rendering and Reconstruction (2509.13938v1)

Abstract: 3D Gaussian Splatting (3DGS) has revolutionized radiance field reconstruction by achieving high-quality novel view synthesis with fast rendering speed, introducing 3D Gaussian primitives to represent the scene. However, 3DGS encounters blurring and floaters when applied to complex scenes, caused by the reconstruction of redundant and ambiguous geometric structures. We attribute this issue to the unstable optimization of the Gaussians. To address this limitation, we present a plug-and-play PDE-based optimization method that overcomes the optimization constraints of 3DGS-based approaches in various tasks, such as novel view synthesis and surface reconstruction. Firstly, we theoretically derive that the 3DGS optimization procedure can be modeled as a PDE, and introduce a viscous term to ensure stable optimization. Secondly, we use the Material Point Method (MPM) to obtain a stable numerical solution of the PDE, which enhances both global and local constraints. Additionally, an effective Gaussian densification strategy and particle constraints are introduced to ensure fine-grained details. Extensive qualitative and quantitative experiments confirm that our method achieves state-of-the-art rendering and reconstruction quality.

• The paper introduces PDEO, a plug-and-play PDE optimization framework that mitigates gradient-induced artifacts to improve rendering and reconstruction quality.
• The methodology reformulates attribute updates as a discretized PDE with a viscous regularization term, solved efficiently using the Material Point Method.
• Experimental results demonstrate superior performance in novel view synthesis and surface reconstruction, with reduced memory usage and enhanced geometric detail.

Plug-and-Play PDE Optimization for 3D Gaussian Splatting: Toward High-Quality Rendering and Reconstruction

Introduction and Motivation

The paper introduces PDEO, a plug-and-play optimization framework for 3D Gaussian Splatting (3DGS), targeting improved stability and quality in both novel view synthesis and surface reconstruction. 3DGS, which represents scenes using explicit 3D Gaussian primitives, has demonstrated high-quality, real-time rendering. However, its optimization process is prone to instability, manifesting as blurring and floaters, especially in complex scenes. The root cause is identified as the disproportionate magnitude of positional gradients for small-scale Gaussians, which leads to abrupt positional updates and suboptimal optimization of other attributes.

To address these issues, the authors propose modeling the 3DGS optimization as a discretized partial differential equation (PDE) and introduce a viscous term inspired by fluid simulation. This term regularizes the optimization dynamics, suppressing abrupt changes and promoting stable convergence. The Material Point Method (MPM) is employed for efficient numerical solution, leveraging both particle and grid representations to enforce global and local constraints.

PDE-Based Formulation and Viscous Regularization

The optimization of 3D Gaussian attributes is reformulated as a time-dependent PDE, where each attribute evolves as a function of time. The velocity of a Gaussian's position is defined by the gradient of the loss function, and its temporal evolution is governed by the PDE:

$$\frac{d\boldsymbol{v}_i^t}{dt} = \frac{\partial \boldsymbol{v}_i^t}{\partial t} + \boldsymbol{v}_i^t \cdot \nabla \boldsymbol{v}_i^t = \frac{\sigma^2}{2} \sum_{\gamma_i^t \in \Gamma_i^t} \nabla \left( \frac{\partial L^t}{\partial \gamma_i^t} \right)^2 + (1-\lambda_g) \nabla \cdot \nabla \boldsymbol{v}_i^t$$

The viscous term $$(1-\lambda_g) \nabla \cdot \nabla \boldsymbol{v}_i^t$$ acts as a local averaging operator, damping abrupt velocity changes and stabilizing the optimization trajectory. This approach preserves the integrity of the original gradient information, in contrast to heuristic methods like gradient clipping or normalization, which can induce information loss.

Figure 1: Optimization of 3D Gaussians. (a) Redundancy of large Gaussians. (b) Ambiguity of small Gaussians. (c) Visualization results.

Material Point Method for Stable Optimization

The MPM framework is adopted to efficiently solve the PDE, treating Gaussians as particles and discretizing the scene into voxel grids. The optimization alternates between Particle-to-Grid (P2G) and Grid-to-Particle (G2P) steps:

• P2G: Particle velocities are attenuated and stored in the voxel grid, capturing local excess motion.
• G2P: The grid velocity field guides particle updates, mixing individual and local average velocities to suppress outlier motions.

This dual representation enables robust enforcement of both local and global constraints, mitigating the instability caused by high-magnitude positional gradients in small Gaussians.

Figure 2: Overview of the proposed PDEO. Gaussians are initialized by COLMAP, optimized via PDE with viscosity, and solved using MPM (P2G/G2P).

Particle Constraints and Densification

To further enhance detail recovery and prevent redundancy, the framework introduces:

• Scale Loss: Penalizes large-scale Gaussians, promoting fine-grained representation of high-frequency details.
• Confidence Loss: Encourages binary opacity, avoiding semi-transparent Gaussians and enforcing the particle hypothesis.
• Gaussian Densification: Uses velocity field-guided cloning and splitting, based on cosine similarity between particle and grid velocities, to adaptively densify regions requiring higher detail.

Experimental Results

Novel View Synthesis

PDEO is integrated with multiple state-of-the-art 3DGS-based methods (e.g., 3DGS, GES, MipGS, RaDeGS, SpecGS) and evaluated on Mip-NeRF360, Tanks&Temples, and ScanNet++ datasets. Quantitative metrics (PSNR, SSIM, LPIPS) consistently show improvements over baselines. Notably, SpecGS+PDEO achieves the highest scores, and memory usage is reduced due to more efficient Gaussian allocation.

Figure 3: Qualitative comparisons on Mip-NeRF360, Tanks&Temples, and ScanNet++ for novel view synthesis. PDEO reduces artifacts and floaters, improving rendering quality.

Figure 4: Visualization of Gaussian ellipsoids. PDEO eliminates floater Gaussians and recovers fine geometric details.

Surface Reconstruction

On DTU and Tanks&Temples, PDEO-enhanced methods yield lower Chamfer Distance errors and higher F1-scores, indicating more accurate and smoother geometry. RaDeGS+PDEO achieves the best reconstruction quality, effectively removing floaters and preserving fine details.

Figure 5: Qualitative comparisons on Tanks&Temples for surface reconstruction. PDEO improves reconstruction quality.

Ablation Studies

Ablation experiments on Mip-NeRF360 demonstrate the necessity of each component. Removing P2G/G2P, scale loss, or confidence loss degrades rendering quality and increases memory usage. The viscosity term and velocity field are critical for stable optimization and artifact suppression.

Figure 6: Ablation of P2G/G2P, scale loss, and confidence loss.

Theoretical and Practical Implications

The PDEO framework provides a principled approach to regularizing the optimization of explicit scene representations, bridging concepts from fluid simulation and radiance field learning. By modeling attribute updates as PDEs and leveraging MPM, the method achieves stable, high-fidelity reconstructions without sacrificing efficiency. The plug-and-play nature allows seamless integration into existing 3DGS pipelines, making it broadly applicable.

The explicit handling of gradient sensitivity and the introduction of viscosity-based regularization offer a robust alternative to heuristic gradient control methods. The approach is theoretically sound, as the viscous term does not alter the optimization solution in the limit, and practically effective, as evidenced by strong empirical results.

Limitations and Future Directions

The current framework does not address particle rotation, which could be incorporated by extending the MPM simulation to rotational attributes. Additionally, the method struggles to relocate Gaussians into regions lacking initial point cloud coverage, suggesting future work on adaptive initialization or global guidance mechanisms.

Conclusion

PDEO establishes a rigorous, physically-inspired optimization paradigm for 3D Gaussian Splatting, yielding state-of-the-art results in both rendering and reconstruction. Its integration of PDE modeling, viscous regularization, and MPM-based numerical solution addresses fundamental limitations of prior approaches, offering a scalable and generalizable solution for high-quality scene representation. Future research may extend the framework to rotational dynamics and more adaptive initialization strategies, further enhancing its applicability in complex real-world scenarios.