EnerGS: Energy-Based Gaussian Splatting with Partial Geometric Priors

Abstract: 3D Gaussian Splatting (3DGS) has been widely adopted for scene reconstruction, where training inherently constitutes a highly coupled and non-convex optimization problem. Recent works commonly incorporate geometric priors, such as LiDAR measurements, either for initialization or as training constraints, with the goal of improving photometric reconstruction quality. However, in large-scale outdoor scenarios, such geometric supervision is often spatially incomplete and uneven, which limits its effectiveness as a reliable prior and can even be detrimental to the final reconstruction. To address this challenge, we model partially observable geometry as a continuous energy field induced by geometric evidence and propose EnerGS. Rather than enforcing geometry as a hard constraint, EnerGS provides a soft geometric guidance for the optimization of Gaussian primitives, allowing geometric information to steer the optimization process without directly restricting the solution space. Extensive experiments on large-scale outdoor scenes demonstrate that, under both sparse multi-view and monocular settings, EnerGS consistently improves photometric quality and geometric stability, while effectively mitigating overfitting during 3DGS training.

• The paper introduces EnerGS, which decouples geometric and photometric optimization using an energy-based regularization framework for 3D Gaussian splatting.
• It employs a partitioned energy field with a Welsch M-estimator, Boltzmann barrier, and weak priors to enforce reliable reconstruction in occupied, free, and unknown regions.
• Experimental results on KITTI and Waymo datasets show improved geometric consistency, minimized free-space leaks, and enhanced generalization compared to baseline methods.

Energy-Based Gaussian Splatting with Partial Geometric Priors: An Expert Analysis

Context and Motivation

Recent advances in novel view synthesis, particularly 3D Gaussian Splatting (3DGS), have elevated the efficiency and fidelity of photorealistic scene reconstruction. 3DGS, which represents a scene using anisotropic 3D Gaussian primitives rendered via differentiable rasterization, achieves real-time performance and high visual quality. Nonetheless, its explicit and discrete representation exhibits well-known degeneracies under sparse observation, particularly for large-scale, unbounded outdoor environments where photometric signals alone are insufficient and geometric priors such as LiDAR are structurally sparse and spatially incomplete. Traditional strategies that rigidly integrate geometric priors have shown limited efficacy and may even degrade performance if the priors are unevenly distributed.

Methodological Contributions

This work introduces EnerGS, a unified energy-based regularization framework for 3DGS that encodes partial geometric priors as a continuous differentiable energy field. EnerGS formulates geometric constraints as a partition of physical trust encompassing three regimes: occupied (LiDAR-confirmed), free (certified empty via LiDAR), and unknown (unobserved) space. The methodology leverages:

• Welsch M-estimator Attraction in Occupied Regions: A robust volumetric attractor mitigates sensitivity to outliers, concentrating Gaussians near observed surfaces without being dominated by sparse or noisy sensor points.
• Boltzmann Barrier in Free Space: Primitives are strictly repulsed from LiDAR-certified free regions using a softplus-based potential, ensuring monotonic physical exclusion and preventing persistent floaters or artifacts.
• High-Variance Weak Prior in Unknown Regions: In unobserved volumes, the energy field relaxes, allowing photometric loss to dominate spline placement and avoiding the suppression of visually plausible geometry absent from LiDAR.

EnerGS decouples the gradient-based optimization of Gaussian means from their appearance, preventing conflicting updates arising in photometrically ambiguous settings. Instead, appearance and covariance are updated via photometric gradients, while positions follow the strictly monotonic geometric energy field, with discrete pruning enforcing hard boundary constraints when necessary.

Theoretical and Practical Properties

EnerGS is analyzed as a constrained dynamical system. Notable theoretical assertions and proofs include:

• Non-existence of Stable Degenerate Solutions in Trusted Free Space: The decoupled geometric descent ensures primitives cannot persist in certified free regions, independent of the photometric landscape, effectively eliminating floaters.
• Lipschitz Regularity of the Energy Field: The energy-induced gradient field is smooth and bounded, suppressing oscillatory artefacts typical of visibility jumps in rasterization-derived fields and ensuring optimization stability.
• Asymptotic Permissiveness in Unobserved Space: The weak prior's influence vanishes with increasing uncertainty. Thus, photometric cues alone determine structure in sensor blind spots without introducing bias or loss of plausible geometry.

Experimental Results

EnerGS was evaluated on the KITTI and Waymo Open datasets, with key findings:

• Superior Geometric Consistency with Photometric Fidelity: EnerGS achieves the highest PSNR and SSIM metrics, and, more importantly, the lowest free-space Leak scores and highest OccCov, reflecting faithful scene occupancy and minimal geometric violations.
• Generalization and Overfitting Suppression: The method consistently yields a smaller train-test PSNR gap compared to baselines, evidencing improved learning of multi-view-consistent geometry rather than memorization of training images.
• Ablation Studies: Removing or altering the design of any regime within the geometric energy field, or reverting to joint optimization, consistently degrades both geometric and photometric metrics, highlighting the necessity of EnerGS' partitioned trust and optimization decoupling.

Implications and Future Directions

EnerGS demonstrates that explicit, spatially adaptive modeling of geometric observability outperforms rigid or uniform regularization in large-scale, real-world scenarios where sensors are inevitably sparse and coverage is incomplete. The energy field framework is both theoretically sound (provable non-existence of degenerate states, monotonic descent) and practically efficient, with negligible runtime overhead atop standard 3DGS pipelines.

Theoretical Implications: The work formalizes the role of sensor trust in 3D volumetric regularization, proposing a testable, extensible methodology for fusing uncertain geometric and photometric evidence.

Practical Implications: By robustly eliminating floaters and preserving LiDAR-blind but image-visible geometry, EnerGS directly benefits applications in autonomous driving, robotics, and digital twin generation, where reconstruction integrity is paramount, and sensor coverage is inherently partial.

Compatibility and Extensions: The method is compatible with dynamic scene modeling, can be integrated with more advanced LiDAR-based pipelines, and generalizes across varied urban environments and sensor densities. There is scope for extending the energy field with higher-order priors, semantics, or temporal dynamics and adapting it to other explicit radiance field representations.

Conclusion

EnerGS presents a comprehensive, theoretically-grounded, and empirically-validated solution for 3DGS reconstruction under partial geometric supervision. By encoding sensor trust into an adaptive energy field and decoupling spatial and appearance optimization, EnerGS achieves state-of-the-art geometric and photometric performance, enhanced generalization, and robust artifact suppression. This work substantiates and advances the framework for principled multi-modal fusion in the next generation of photorealistic scene representations (2604.26238).