You Only Gaussian Once: Controllable 3D Gaussian Splatting for Ultra-Densely Sampled Scenes

Abstract: 3D Gaussian Splatting (3DGS) has revolutionized neural rendering, yet existing methods remain predominantly research prototypes ill-suited for production-level deployment. We identify a critical "Industry-Academia Gap" hindering real-world application: unpredictable resource consumption from heuristic Gaussian growth, the "sparsity shield" of current benchmarks that rewards hallucination over physical fidelity, and severe multi-sensor data pollution. To bridge this gap, we propose YOGO (You Only Gaussian Once), a system-level framework that reformulates the stochastic growth process into a deterministic, budget-aware equilibrium. YOGO integrates a novel budget controller for hardware-constrained resource allocation and an availability-registration protocol for robust multi-sensor fusion. To push the boundaries of reconstruction fidelity, we introduce Immersion v1.0, the first ultra-dense indoor dataset specifically designed to break the "sparsity shield." By providing saturated viewpoint coverage, Immersion v1.0 forces algorithms to focus on extreme physical fidelity rather than viewpoint interpolation, and enables the community to focus on the upper limits of high-fidelity reconstruction. Extensive experiments demonstrate that YOGO achieves state-of-the-art visual quality while maintaining a strictly deterministic profile, establishing a new standard for production-grade 3DGS. To facilitate reproducibility, part scenes of Immersion v1.0 dataset and source code of YOGO has been publicly released. The project link is https://jjrcn.github.io/YOGO/.

• The paper introduces deterministic budget control that allocates Gaussian primitives by region and matches user-defined targets, enabling predictable memory use and region-of-interest refinement.
• YOGO combines radiometric sensor registration, availability filtering, and optimization improvements to preserve quality across heterogeneous sensors, reaching 27.73 dB PSNR and improving roaming-view Qalign to 3.7771.
• The Immersion v1.0 benchmark uses ultra-dense indoor captures to test physical fidelity, while experiments show that models with about 1.5 million points can outperform larger baselines and that added points yield diminishing returns.

YOGO (You Only Gaussian Once) is a system-level framework for 3D Gaussian Splatting (3DGS) that targets production deployment, addressing what the authors identify as an "Industry-Academia Gap": unpredictable resource consumption from heuristic densification, benchmark sparsity that rewards hallucination over physical fidelity, and radiometric inconsistency across heterogeneous sensors. The paper contributes a deterministic budget controller, an availability-registration protocol for multi-sensor fusion, a set of optimization refinements, and a new ultra-dense indoor benchmark, Immersion v1.0 (2604.21400).

Deterministic budget control

The core methodological move is to replace 3DGS's heuristic densification thresholds with a feedback-loop controller that treats Gaussian growth as a budgeted allocation problem. The scene is partitioned into $M$ disjoint spatial polygons, each assigned a target primitive count $N^{target}_m$. At each of the $K$ scheduled densification events, YOGO applies a prune-then-densify sequence in which the per-step quota is computed from the remaining gap $\Delta N_m(k)$ divided by the number of events left, and the top-$Q$ Gaussians by accumulated gradient magnitude are selected within each polygon.

Two properties follow directly. First, the final primitive count converges to $\sum_m N^{target}_m$, eliminating trial-and-error tuning for hardware memory limits — a claim the experiments support by evaluating three fixed-budget variants (roughly 1.5M, 3.5M, and 5.8M points) whose counts match their budgets. Second, polygon-level allocation enables region-of-interest refinement: dense budgets for complex foreground regions and aggressive compression elsewhere. This contrasts with post-hoc compression pipelines (Compressed 3DGS, Mini-Splatting) that require an unconstrained training phase before pruning or quantization.

Availability-registration multi-sensor fusion

For heterogeneous sensor inputs, the authors propose a three-stage hierarchical protocol: foundation training on the primary sensor, radiometric registration of downsampled auxiliary data via frozen-anchor optimization of per-view affine color transforms $\mathbf{A}_j = [\mathbf{G}_j | \mathbf{b}_j]$, then hybrid training restricted to auxiliary frames passing an availability score. The score penalizes deviation of $\mathbf{A}_j$ from identity across three terms — peak channel-wise gain deviation, cross-channel crosstalk, and luminance bias — with frames exceeding threshold $\tau$ rejected as polluted data.

The ablation is instructive: naive direct fusion degrades PSNR relative to single-sensor training (27.34 vs. 27.73 dB), while A.R. fusion at $\tau=0.15$ preserves validation metrics and yields the best roaming-test Qalign (3.7771 vs. 3.7142 single-sensor). Notably, on the validation set multi-sensor dense (MSD) slightly trails single-sensor dense (SSD), but the ordering inverts on out-of-distribution roaming views — evidence that auxiliary sensors act as regularization against trajectory overfitting rather than improving in-distribution interpolation.

Solid optimization suite

Three targeted modifications address high-frequency detail and model compactness. Area-normalized gradient accumulation aggregates per-pixel absolute gradients normalized by cross-view pixel footprint, avoiding both directional cancellation (as in coordinate-averaged gradients) and the large-Gaussian bias of pixel-aware schemes. Maximum effective opacity pruning removes primitives whose opacity weighted by accumulated transmittance is negligible in every view — targeting occluded Gaussians invisible to standard $N^{target}_m$0-thresholding. Principal axis densification unifies split and clone into a directional triplet expansion along each Gaussian's major scaling axis, with energy-conserving attenuation of scale and opacity. The component-wise ablation shows monotone compounding gains from the DBC baseline (27.50/3.6292 Qalign) to the full suite (27.73/3.6839), with the gradient scheme and principal-axis densification contributing most jointly.

Immersion v1.0 dataset

Immersion v1.0 comprises seven complex indoor scenes captured with an Insta360 X5 panoramic rig, a four-unit Osmo array, and a SHARE SLAM S20 hybrid LiDAR-visual system (~200K points/s). A six-loop trajectory strategy (three heights, bi-directional loops, close-up augmentation) produces ~30K synchronized frames per scene — roughly 15× ScanNet++ and 100× Tanks & Temples — with ~2.55M initial points per scene. The authors introduce two density metrics: Image Density per Square Meter (IDSM ≈ 500, about 4× ScanNet++) and grid-based Scene Coverage (72% of 0.2 m cells containing more than six poses, versus 16% for ScanNet++). The stated purpose is to break the "sparsity shield": with saturated coverage, evaluation measures physical fidelity rather than view-interpolation capacity. The main concession here is scale — seven scenes is small, which the authors attribute to the computational cost of processing saturated captures; only part of the dataset and code are released.

Experimental results

Evaluation uses three tracks (SSS ~2K images, SSD >20K, MSD ~30K) with a held-out validation set of 100 trajectory frames and a 200-frame roaming test set scored by Qalign, a no-reference LMM-based perceptual metric. The authors validate Qalign against reference metrics on the validation set, reporting correlations of r = 0.95 with PSNR, r = 0.94 with SSIM, and r = −0.95 with LPIPS (all p < 0.001).

Against 3DGS, AbsGS, Mip-Splatting, Scaffold-GS, and Perceptual-GS, YOGO leads on all tracks. Two results stand out. On SSD and MSD, the most constrained variant YOGO1 (~1.5M points) exceeds AbsGS's quality despite AbsGS using roughly 4.3–5.3M points — e.g., 27.73 vs. 27.62 PSNR on SSD. Second, performance saturates sharply with budget: on SSS, doubling from YOGO1 to YOGO2 gains 0.13 Qalign while the next +2M points add only 0.04, quantifying diminishing returns under deterministic control. A caveat weakens the comparison's breadth: Mip-Splatting, Scaffold-GS, and Perceptual-GS could not be run on the SSD/MSD tracks due to OOM failures and prohibitive hyperparameter recalibration, so state-of-the-art claims on the dense tracks rest on comparisons with 3DGS and AbsGS only.

Limitations and open questions

Beyond the small scene count, several assumptions deserve note. The availability score presumes radiometric distortion is well-captured by a per-view affine transform; geometric misalignment beyond what this filtering catches may leak into training. The budget controller assumes user-specified per-polygon targets, leaving automatic allocation unaddressed — the authors list semantic-aware budget assignment as future work. Evaluation relies partly on Qalign without ground truth; although its correlation with reference metrics is strong on validation data, its reliability on genuinely novel viewpoints remains inferred rather than directly verified. Finally, whether deterministic budgeting generalizes to unbounded or dynamic scenes is explicitly left open.

Conclusion

YOGO reframes 3DGS densification as a closed-loop resource-allocation problem, achieving strictly bounded primitive counts with competitive or superior rendering quality, and pairs it with Immersion v1.0, a saturation-oriented benchmark that shifts evaluation toward physical fidelity and cross-sensor robustness. The strongest empirical claims — sub-1.5M-point models beating larger baselines and multi-sensor training improving out-of-distribution generalization — are well supported on the released tracks, though the limited scene count and partial baseline coverage temper their generality.