REUrbanGS: Robust Urban 3D Gaussian Splatting

• REUrbanGS is a robust and efficient 3D Gaussian Splatting approach designed for urban-scale scene reconstruction with real-time rendering under memory constraints.
• It employs scene partitioning, visibility-based image selection, and a controllable level-of-detail strategy to manage large urban datasets and mitigate appearance variations.
• The framework integrates fine-grained appearance transformations, depth and scale regularization, and antialiasing to enhance photometric consistency and reconstruction quality.

REUrbanGS, short for “Robust and Efficient 3D Gaussian Splatting for Urban Scene Reconstruction,” is a 3D Gaussian Splatting framework for urban-scale scene reconstruction that is explicitly designed to preserve fast reconstruction and real-time rendering while remaining robust to appearance variations across multi-view captures. Its central design combines scene partitioning, visibility-based image selection, in-partition prioritized densification, a controllable level-of-detail strategy, a fine-grained appearance transform module, and auxiliary enhancement modules such as depth regularization, scale regularization, and antialiasing, with the stated goal of reconstructing large, complex urban scenes under strict memory budgets while outperforming previous approaches in both efficiency and quality (Yuan et al., 30 Jul 2025).

1. Scope, problem setting, and design objectives

REUrbanGS targets large, complex urban scenes reconstructed from multi-view RGB images with camera poses from SfM, specifically COLMAP outputs that also provide a 3D point cloud and 2D feature associations (Yuan et al., 30 Jul 2025). The framework is motivated by two failure modes of vanilla 3DGS in urban settings. The first is scalability: explicit Gaussians scale poorly with scene size, and millions of primitives lead to GPU OOM, slow training, and non-real-time rendering. The second is robustness: urban images are often captured over long periods, in different weather, lighting, and with multiple cameras, so standard 3DGS overfits these appearance inconsistencies by spawning Gaussians that only explain a few views, producing floaters and artifacts (Yuan et al., 30 Jul 2025).

A common misconception is that large-scene Gaussian splatting is primarily a compression problem applied after training. REUrbanGS departs from that view by building its LOD hierarchy during training rather than relying on a heavy post-compression and fine-tuning stage (Yuan et al., 30 Jul 2025). Another misconception is that appearance variation can be handled only with image-space post-processing. REUrbanGS instead models appearance in Gaussian space through per-Gaussian and per-image embeddings, so robustness to multi-session capture is treated as part of the scene representation itself (Yuan et al., 30 Jul 2025).

The framework is intended for static urban scenes. Dynamic objects are removed via 2D detection and segmentation rather than modeled explicitly, which places REUrbanGS in a different category from dynamic urban 3DGS systems such as OmniRe, which reconstructs vehicles, pedestrians, cyclists, and other dynamic actors as independently controllable scene-graph nodes (Chen et al., 2024).

2. System architecture and scene partitioning

The pipeline begins with multi-view RGB images and camera poses from COLMAP, then divides the scene horizontally into axis-aligned 3D blocks, for example $180\,\mathrm{m}$ or $400\,\mathrm{m}$ partitions (Yuan et al., 30 Jul 2025). Camera assignment is not based solely on camera-center location. If an image lies outside a partition but may still observe it, REUrbanGS computes a point-based visibility score from SfM features. For an unassigned image $I_i$ and partition $j$, the visibility is defined as

$\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$

where $V_i$ is the convex hull area of the full projected 3D point cloud in image $I_i$, and $V_{ij}$ is the convex hull area of feature points associated with 3D points inside partition $j$ (Yuan et al., 30 Jul 2025). Images whose visibility ratio exceeds a threshold such as $1/6$ are assigned to that partition (Yuan et al., 30 Jul 2025).

This point-based visibility formulation is important because SfM feature points are inherently occlusion-aware. It allows REUrbanGS to keep cross-boundary views that matter while avoiding large numbers of irrelevant cameras (Yuan et al., 30 Jul 2025). After initial assignment, partitions are rebalanced by merging partitions with too few images with their smallest neighbor and splitting partitions with too many images, so training load is distributed more uniformly (Yuan et al., 30 Jul 2025).

The full system can be summarized as follows.

Stage	Function	Main mechanisms
Input	Scene initialization	Multi-view RGB, COLMAP poses, 3D point cloud, 2D feature associations
Scene and data division	Scalable partitioning	Horizontal tiling, point-based visibility, partition rebalancing
Parallel partition training	Local urban 3DGS optimization	Prioritized densification, LOD generation, appearance transform, depth and scale regularization, antialiasing, AbsGS
Rendering	Real-time view synthesis	Distance-based partition LOD, view-frustum culling, tile-based culling, optional appearance modulation

The implementation is based on gsplat, and each partition is trained as a separate 3DGS or REUrbanGS model. Partitions can be distributed across GPUs or time-multiplexed, while any given GPU needs to hold only a small subset of partition models. The combination of partitioning, visibility filtering, in-partition densification, budgets, and low-resolution training at coarse LODs keeps per-GPU memory well below $400\,\mathrm{m}$0 even for BigCity (Yuan et al., 30 Jul 2025).

3. Gaussian formulation, regularization, and budgeted optimization

REUrbanGS inherits the standard 3DGS representation in which each Gaussian is

$400\,\mathrm{m}$1

with center $400\,\mathrm{m}$2, covariance $400\,\mathrm{m}$3, color $400\,\mathrm{m}$4, and opacity $400\,\mathrm{m}$5 (Yuan et al., 30 Jul 2025). Rendering follows depth sorting and alpha blending: $400\,\mathrm{m}$6 where $400\,\mathrm{m}$7 is a pixel, $400\,\mathrm{m}$8 is the projected 2D Gaussian, and $400\,\mathrm{m}$9 are Gaussians covering the pixel (Yuan et al., 30 Jul 2025).

The framework’s first major optimization device is in-partition prioritized densification. Rather than densifying all Gaussians under a global rule, REUrbanGS defines a distance-dependent densification threshold $I_i$0 for each Gaussian based on its distance $I_i$1 to the current partition, a maximum distance $I_i$2, and a scaling factor $I_i$3 (Yuan et al., 30 Jul 2025). Gaussians inside the partition receive a lower threshold and are easier to densify; Gaussians far outside are suppressed; boundary Gaussians are interpolated smoothly (Yuan et al., 30 Jul 2025). The reported effect is a reduction in Gaussian count by up to $I_i$4 and a training-time reduction of $I_i$5 with negligible quality change (Yuan et al., 30 Jul 2025).

The second major device is a set of geometry and stability regularizers. Depth regularization uses Depth Anything V2 to predict per-image depth maps, aligns them to metric depth using the SfM point cloud, and applies

$I_i$6

during training (Yuan et al., 30 Jul 2025). This discourages floaters and improves depth consistency in weakly constrained regions. Scale regularization constrains both absolute scale and anisotropy. The anisotropy ratio is

$I_i$7

with $I_i$8 in experiments, and Gaussians that are too large or too stretched are penalized to reduce popping and depth-ordering artifacts (Yuan et al., 30 Jul 2025).

REUrbanGS further integrates Mip-Splatting for antialiasing and AbsGS for fine-detail enhancement, especially at the highest LOD level (Yuan et al., 30 Jul 2025). The full per-partition loss augments the 3DGS photometric loss with embedding similarity regularization, opacity offset regularization, depth loss, and scale regularizers. Hyperparameters reported in the paper include $I_i$9, $j$0, $j$1, and a depth weight $j$2 that starts at $j$3 and decays exponentially to $j$4 (Yuan et al., 30 Jul 2025).

4. Bottom-up level of detail and real-time rendering

A defining feature of REUrbanGS is its controllable, bottom-up LOD strategy. For each partition, the method defines $j$5 LOD levels with three monotone sequences: $j$6

$j$7

$j$8

where $j$9 is the densification budget, $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$0 is the densification interval, and $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$1 is the image downsampling factor at level $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$2 (Yuan et al., 30 Jul 2025). Lower LODs use smaller budgets, longer densification intervals, and lower-resolution images; finer levels refine the coarse model rather than replacing it.

The paper gives a concrete Rubble example: $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$3 Training proceeds bottom-up: levels 1 and 2 use 15k iterations each, and level 3 uses 30k iterations, with densification enabled in the first half (Yuan et al., 30 Jul 2025). Densification intervals and iteration counts are further scaled by $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$4, where $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$5 is the number of images in the partition (Yuan et al., 30 Jul 2025).

At render time, REUrbanGS first identifies visible partitions by frustum intersection, then selects an LOD for each visible partition according to camera distance. An example ablation setting on JNU-ZH uses the third level below $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$6, the second between $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$7 and $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$8, and the first above $\mathrm{visibility}(I_i,j)=\frac{V_{ij}}{V_i},$9 (Yuan et al., 30 Jul 2025). Within each visible partition, tile-based culling from StopThePop drops low-contribution Gaussians, further reducing rasterization cost (Yuan et al., 30 Jul 2025).

This design gives REUrbanGS direct control over both training and rendering resources. Budgets $V_i$0 cap Gaussian growth during training, so memory control is not deferred to a later compression stage. At inference, LOD plus tile culling reduce Gaussian counts by $V_i$1 relative to full detail, and REUrbanGS with LOD typically runs at $V_i$2 FPS at test resolution on a single A100-80G while maintaining higher PSNR and SSIM than CityGaussian and Hierarchical-3DGS (Yuan et al., 30 Jul 2025).

5. Appearance transformation and multi-session robustness

Urban-scale image collections frequently contain appearance changes across months, weather conditions, illumination regimes, and camera systems. REUrbanGS treats this as a representational problem rather than as nuisance noise. Its fine-grained appearance transform assigns a $V_i$3-dimensional embedding to each Gaussian and a $V_i$4-dimensional embedding to each image, concatenates them, and feeds them to a small MLP with one hidden layer of $V_i$5 channels, ReLU activation, and sigmoid output (Yuan et al., 30 Jul 2025).

The MLP predicts per-image, per-Gaussian offsets $V_i$6 and $V_i$7, yielding effective attributes

$V_i$8

This is more fine-grained than a purely per-image latent code because the same image embedding can modulate different Gaussians differently (Yuan et al., 30 Jul 2025). The design also enables post-hoc appearance control: given a reference image embedding, new views can be synthesized to match that reference appearance, such as “under maintenance, sunset lighting” versus “maintenance completed, overcast” (Yuan et al., 30 Jul 2025).

Two regularizers stabilize this module. The first is a similarity regularizer that encourages neighboring Gaussians to have similar embeddings via a cosine loss weighted by

$V_i$9

thereby suppressing localized color blotches (Yuan et al., 30 Jul 2025). The second is an opacity offset regularizer

$I_i$0

which discourages the model from abusing opacity changes to fit appearance and forces most variation into color offsets (Yuan et al., 30 Jul 2025).

Optimization details reported for this module include a learning rate of $I_i$1 decayed exponentially to $I_i$2, and, every 50 iterations, sampling $I_i$3 Gaussians with $I_i$4 nearest neighbors for the similarity loss (Yuan et al., 30 Jul 2025). Evaluation follows NeRF-W practice: for each test image, the embedding is optimized on half the image, metrics are computed on the other half, then halves are swapped and averaged (Yuan et al., 30 Jul 2025).

6. Empirical performance and ablation evidence

REUrbanGS is evaluated on Mega-NeRF scenes such as Rubble and Building, UrbanScene3D scenes such as Residences, Sci-Art, and Campus, and two self-captured drone datasets, JNU-ZH and BigCity (Yuan et al., 30 Jul 2025). JNU-ZH contains about $I_i$5 drone images over roughly $I_i$6, collected over 3 months with 2 different cameras; BigCity contains about $I_i$7 drone images from 5 cameras over roughly $I_i$8 (Yuan et al., 30 Jul 2025). Metrics are PSNR, SSIM, LPIPS, FPS, and Gaussian count in millions (Yuan et al., 30 Jul 2025).

Representative LOD results are as follows.

Scene	REUrbanGS (LOD)	Selected comparator(s)
Rubble	SSIM 0.814, PSNR 27.03, LPIPS 0.245, $I_i$9M, FPS $V_{ij}$0	CityGaussian: PSNR 24.90, $V_{ij}$1M, FPS 105.2; Hierarchical-3DGS: PSNR 23.38, FPS 57.4
JNU-ZH	SSIM 0.816, PSNR 25.71, LPIPS 0.240, $V_{ij}$2M, FPS 63.9	CityGaussian: PSNR 22.33, $V_{ij}$3M, FPS 69.2; Hierarchical-3DGS: PSNR 21.12, $V_{ij}$4M, FPS 34.6
BigCity	SSIM 0.838, PSNR 26.41, LPIPS 0.231, $V_{ij}$5M, FPS 73.0	CityGaussian: PSNR 22.24, $V_{ij}$6M, FPS 122.5; Hierarchical-3DGS: PSNR 23.17, $V_{ij}$7M, FPS 3.2

These numbers illustrate the paper’s central trade-off: REUrbanGS does not always minimize Gaussian count or maximize raw FPS, but it is designed to sustain higher reconstruction quality under memory-constrained city-scale conditions (Yuan et al., 30 Jul 2025). The paper also notes that some baselines cannot run on BigCity within $V_{ij}$8 GPU memory without aggressive pruning or parameter tuning, whereas REUrbanGS runs within $V_{ij}$9 for LOD (Yuan et al., 30 Jul 2025).

Ablations show that the system’s components are complementary rather than redundant. Removing point-based visibility increases the number of training cameras by $j$0 without improving quality; removing the appearance transform reduces PSNR and increases LPIPS, while also slightly increasing Gaussian count because the model over-densifies to explain view-specific colors; removing depth regularization causes visually important floaters; removing scale regularization produces popping and smears near large or thin structures; and tile-based culling yields large FPS gains with negligible metric change (Yuan et al., 30 Jul 2025). This evidence supports the interpretation of REUrbanGS as a systems-level urban 3DGS design rather than a single-module modification.

7. Limitations and position within the urban Gaussian splatting literature

The paper identifies two explicit limitations. First, REUrbanGS depends on accurate SfM poses, and noisy or inconsistent poses can degrade reconstruction quality, especially when partitions do not align well with scene geometry (Yuan et al., 30 Jul 2025). Second, its LOD switching is non-incremental: separate full models are maintained per level, increasing storage and adding some selection overhead (Yuan et al., 30 Jul 2025). The paper also notes implicit limitations: a static-scene assumption, degraded performance under extremely sparse or highly unbalanced view coverage, and substantial engineering complexity because the system combines partitioning, LOD, appearance modeling, depth and scale regularization, antialiasing, fine-detail enhancement, and transient removal (Yuan et al., 30 Jul 2025).

Within the broader literature, REUrbanGS occupies the branch of urban 3DGS work centered on scalable static-scene reconstruction with strong resource control. StreetSurfGS emphasizes planar-based octree representations, segmented training, guided smoothing, and dual-step matching for long, narrow street trajectories (Cui et al., 2024). UrbanGS emphasizes geometric accuracy through Depth-Consistent D-Normal Regularization, adaptive confidence weighting, Spatially Adaptive Gaussian Pruning, and a unified partitioning and view assignment scheme (Li et al., 2 Feb 2026). MetroGS adopts distributed 2D Gaussian Splatting, pointmap-assisted dense initialization, progressive hybrid geometric optimization, and depth-guided appearance modeling for large-scale geometric fidelity (Chen et al., 24 Nov 2025). GS4City extends urban Gaussian splatting toward hierarchical semantics by projecting LoD3 CityGML priors into image space and learning per-Gaussian identity codes for structured, queryable city-model semantics (Zhang et al., 13 Apr 2026). OmniRe, by contrast, addresses dynamic urban digital twins through a Gaussian scene graph with rigid, SMPL-based, and deformable actor nodes for vehicles, pedestrians, cyclists, and other dynamic objects (Chen et al., 2024).

This suggests a useful taxonomy. REUrbanGS is primarily a robust-and-efficient static urban reconstruction framework; UrbanGS and MetroGS push more strongly on geometric fidelity; GS4City pushes on structured semantics; OmniRe pushes on dynamic controllability. A plausible implication is that later “urban digital twin” systems can be viewed as adding geometry-centric, semantic, or dynamic modules on top of the large-scene resource-control principles that REUrbanGS makes explicit (Yuan et al., 30 Jul 2025).