SharpSplat: Edge-Regularized 3D Gaussian Splatting for High Fidelity Urban Building Reconstruction from UAV images

Abstract: Reconstructing high-fidelity 3D building models from UAV imagery is essential for large-scale digital twin development. However, existing 3D Gaussian Splatting (3DGS) techniques often struggle with building facades, failing to capture sharp geometric transitions. To address this, we propose a semantic edge regularization framework that supervises 3DGS to produce crisp architectural boundaries. Our method leverages SAM 3 to generate precise building masks, from which we extract architecturally significant edges. During training, we align rendered image gradients with these extracted edges, forcing the Gaussians to converge into sharp structural geometries. Evaluations across campus environments, dense urban centers, and custom residential datasets demonstrate significant improvements in edge fidelity without requiring architectural modifications to the 3DGS pipeline. Our approach proves robust across diverse building types, roof geometries, and urban densities.

• The paper introduces an edge-regularized extension to 3D Gaussian Splatting that enhances the reconstruction of sharp architectural boundaries.
• It leverages SAM3 for semantic building mask extraction and integrates an edge alignment loss to guide the optimization process.
• Experimental results demonstrate improved PSNR, SSIM, and LPIPS metrics in urban scenes without increasing computational cost.

SharpSplat: Edge-Regularized 3D Gaussian Splatting for High Fidelity Urban Building Reconstruction from UAV Images

Introduction

Urban-scale 3D reconstruction from UAV imagery is critical for digital twins, infrastructure monitoring, and urban planning. While 3D Gaussian Splatting (3DGS) has supplanted previous photogrammetric and radiance-field-based methods due to explicit primitive representation and real-time rendering capabilities, its application to buildings exposes a key shortcoming: the inability to accurately reproduce sharp architectural boundaries. Standard 3DGS optimizes for photometric fidelity globally, but does not enforce local geometry constraints at semantic boundaries, resulting in models where facades and edges appear blurred, and fine details such as window frames and corners are lost.

SharpSplat addresses this challenge by integrating semantic edge supervision into the 3DGS optimization process––without requiring architectural modifications. Leveraging SAM3 for precise building mask extraction, and subsequently isolating architecturally significant edges, the method introduces an edge alignment loss that drives Gaussians during training to produce crisp, well-localized 3D boundaries. This essay summarizes SharpSplat's methodology, experimental validation, and its implications for the field.

Methodology

SharpSplat integrates edge supervision into 3D Gaussian Splatting in three distinct stages: SAM-based semantic edge extraction, rendered edge computation, and edge alignment via loss regularization.

After multi-view UAV images are processed by SAM3 (Segment Anything Model 3), text-prompt-driven binary building masks are obtained. Edge maps are extracted from the original images via Sobel filtering and restricted to architectural regions by elementwise multiplication with the building masks. This pipeline preserves semantically relevant boundaries and fine image gradients necessary for architectural fidelity.

Figure 1: Semantic edge supervision pipeline with SAM3 for building mask extraction, edge detection, and edge-driven alignment during training.

During 3DGS optimization, rendered views of the current model are similarly processed into edge maps via grayscale conversion and Sobel filtering. Both the ground-truth (SAM3) and rendered edge maps are normalized. A pixelwise L1 loss between these two edge representations is integrated into the training loss:

$$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{RGB}} + \lambda_{\text{SSIM}} \mathcal{L}_{\text{SSIM}} + \lambda_{\text{edge}} \mathcal{L}_{\text{edge}}$$

where $\mathcal{L}_{\text{edge}}$ is the mean absolute difference in edge strength at each pixel, and all edge detection steps are implemented via efficient Conv2D operations. By precomputing SAM3 edge masks offline and loading them during training, the approach maintains computational and memory efficiency. No architectural modifications to the 3DGS framework are required.

Experimental Evaluation

The method is evaluated on both public and custom UAV datasets representative of dense urban centers (Art Sci), institutional campuses (PolyTech), and varied residential topologies (Gehukheda). Comparisons are made against standard 3DGS, surface-aligned splatting (2DGS), and SuGaR. The evaluation metrics include PSNR, SSIM, and LPIPS.

Results indicate consistent improvements in edge fidelity and visual sharpness without degradation in global reconstruction metrics. Across scenes, SharpSplat outperforms baselines in clarity of architectural boundaries. The method is robust to occlusion, diverse roof geometries, and variable urban densities, and is effective across both short (7K) and longer (15K) training schedules.

Figure 2: Qualitative comparisons across representative scenes. SharpSplat produces sharper building edges and details compared to SOTA approaches.

Figure 3: In the PolyTech scene, SharpSplat (right) preserves window structure, facade alignment, and sharp corners versus baseline 3DGS (left).

Figure 4: The Art Sci scene demonstrates significant improvement in window frame clarity with edge supervision.

Numerically, SharpSplat achieves up to +0.2 dB gain in PSNR, marginal increases in SSIM, and improved LPIPS for the focused regions of interest. Importantly, these improvements are realized without any increase in model memory requirements or changes in inference speed, as edge supervision operates solely during training.

Analysis and Discussion

SharpSplat's contribution is orthogonal to prior geometric and semantic enhancements for 3DGS-based reconstruction. Notably, while methods such as EdgeGaussians (Pepe et al., 2024) and DN-Splatter [WACV 2025] attempt to impose edge-awareness via fundamental changes to primitive generation or dependence on unreliable depth/normal estimation, SharpSplat only leverages robust image-based semantics and simple edge filters. Its supervision occurs strictly at the level of appearance, not geometry.

The approach's primary limitation is the accuracy of semantic masks produced by SAM3. In scenes with ambiguous text prompts or occlusions, errors in mask generation can propagate to edge extraction, potentially misguiding the regularization loss. Additionally, the weighting hyperparameter $\lambda_{\text{edge}}$ necessitates per-scene tuning; although the optimal range is narrow, further automation or adaptive schemes could be beneficial.

Combining SharpSplat's appearance-based supervision with geometric priors employed by methods such as 2DGS or PG-SAG represents a promising direction for further improving both structural accuracy and visual fidelity in large-scale 3D reconstruction. Adapting the edge alignment paradigm to other Gaussian-based representations, or extending it to temporal or multimodal supervision, is also plausible.

Conclusion

SharpSplat introduces semantic edge supervision as a lightweight, architecture-agnostic regularization mechanism for improving the fidelity of 3D Gaussian Splatting-based urban reconstructions. By aligning rendered image gradients with robust, text-guided semantic edge maps, the approach yields sharper, more accurate architectural boundaries while minimally impacting efficiency. This framework enhances the quality of 3D digital twins for urban analytics, and provides a foundation for further integration of semantic and geometric cues in large-scale inverse graphics pipelines.