HiGS: A Hierarchical Rendering Architecture for Real-Time 3D Gaussian Splatting

Abstract: 3D Gaussian Splatting (3DGS) has become the standard for real-time novel view synthesis on commodity GPUs. Its pipeline ties spatial partitioning and rasterization to one tile size, yet the two pull in opposite directions: partitioning, which bins and depth-sorts gaussians, grows cheaper with larger tiles, while rasterization gets cheaper with smaller ones. Prior acceleration work reduces the cost of individual stages but keeps both locked to that single scale, where a few dense tiles dominate frame time. We present Hierarchically Tiled Gaussian Splatting (HiGS), which gives each its own scale: partitioning runs over coarse macro-tiles, while rasterization runs over the fine render tiles within them. Rasterization work is then issued in proportion to the gaussians in each macro-tile rather than per tile, so dense regions spread across many parallel units instead of serializing through one. Across tested scenes, HiGS renders up to 15.8x faster than the original 3DGS and outperforms every other rasterizer we evaluate, while preserving exact front-to-back alpha compositing.

• The paper presents a two-level hierarchical pipeline that decouples coarse spatial partitioning (macro-tiles) from fine rasterization to optimize load balancing.
• It reduces computational overhead with segmented sorting and density-proportional work distribution, achieving up to 15.8× speedup over conventional methods.
• The approach maintains numerical robustness using fp16 arithmetic, preserving high image fidelity while enabling scalable real-time rendering on commodity GPUs.

HiGS: A Hierarchical Rendering Architecture for Real-Time 3D Gaussian Splatting

Introduction and Motivation

Three-dimensional Gaussian Splatting (3DGS) has quickly become the canonical method for real-time neural scene reconstruction, leveraging the strength of explicit, differentiable representations atop commodity GPU rasterization. The standard pipeline, as typified by gsplat, involves projecting gaussians to the image plane, associating them with render tiles, depth sorting, and performing per-tile rasterization. While this factorization aligns well with modern GPU hardware, it induces an undesirable computational trade-off: partitioning and depth sorting favor larger tile sizes (reducing overlap counts and sorting cost), whereas rasterization benefits from smaller tiles (minimizing redundant per-pixel gaussian evaluations).

HiGS ("Hierarchically Tiled Gaussian Splatting") directly addresses the divergence in optimal tile sizes for partitioning versus rasterization. By decoupling the coarse-grained spatial partitioning (macro-tiles) from fine-grained rasterization (render tiles), HiGS restructures the work decomposition to be density-proportional rather than area-proportional. This eliminates the "tail effect" whereby a handful of dense tiles throttle the rasterizer's throughput in one-block-per-tile dispatch schemes.

Figure 1: HiGS renders a 5.8M-gaussian scene at 1,124 FPS (1080p), with much more even load balancing per work unit compared to gsplat.

Hierarchical Work Decomposition

HiGS defines a two-level spatial hierarchy: macro-tiles group together $8\times4$ render tiles, collectively spanning $64\times32$ pixels. Partitioning, binning, and sorting are conducted at macro-tile granularity, while rasterization is parallelized over the constituent render tiles. Each macro-tile's sorted list of gaussians is subdivided into fixed-size batches (default 1,024), each independently processed and assigned to one CUDA CTA. The assignment of rasterization work is thus strictly density-proportional, and no single tile can bottleneck the pipeline.

Figure 2: HiGS's hierarchical pipeline: partitioning and sorting over macro-tiles, rasterization over render tiles.

This approach yields two key architectural effects:

• Segmented Sorting: Sorting of (macro-tile, gaussian) pairs replaces global sorting over composite (tile, depth) keys, reducing both overall key width and memory traffic.
• Dynamic Load Balancing: Dense regions spawn proportionally more processing units; render tiles are dynamically assigned to worker groups, mitigating per-tile throughput variance.

Work Imbalance and Load Balancing

In conventional 3DGS, assigning a thread block per render tile can result in substantial work imbalance due to high variance in the number of overlapping gaussians per tile. HiGS alleviates this by grouping render tiles into macro-tiles and spawning more batches (and thus more work units) in dense macro-tiles.

Figure 3: Left: 3DGS incurs load imbalance and SM under-utilization; Right: HiGS's macro-tile batches achieve bounded and concurrent work allocation.

This density-proportional allocation directly eliminates the persistent tail latency affecting conventional pipelines.

Per-Stage Computational Profile

HiGS produces systematic reductions in expensive pipeline stages through its hierarchical design:

• Pair Count Reduction: Conservative macro-tile ellipse tests reduce tile-gaussian pair counts by up to 92% at 4K compared to gsplat with improved binning, essentially lowering radix sort input volume and partitioning cost.
• Sort Cost Reduction: Segmented sorting on 32-bit depth over macro-tile segments achieves $42$–$73\times$ faster performance than global 64-bit tile-depth composite sort.
• Resolution-Controlled Frame Time: The frame-time scaling with resolution is attenuated—growing only $1.6\times$ from 1080p to 4K versus $2.8\times$ for conventional approaches—by bounding per-unit work irrespective of screen area.

  Figure 4: Per-stage timing breakdown for $8\times8$ and $16\times16$ tiles. HiGS exhibits stable performance profile across tile sizes, with preprocessing-dominated cost on large scenes and rasterization-dominated cost on small scenes.

Empirical Performance and Scaling

Evaluated across seven Mip-NeRF~360 scenes (1.2–6.1M gaussians) and the nvcampus park scene scaled up to 75M gaussians, HiGS consistently outperforms all prior state-of-the-art rasterizers, including strong baselines such as FlashGS, Faster-GS, Speedy-Splat, StopThePop, and TC-GS.

Key results:

• Median frame time improvements: HiGS yields up to $15.8\times$ speedup over 3DGS and $2.2\times$ over FlashGS, with consistent superiority at both 1080p and 4K.
• FPS rates: Achieves 1316 FPS on a 6.1M-gaussian scene at 1080p and 949 FPS at 4K, with leading numbers in all reported benchmarks.
• Linear scaling with gaussian count: Frame time grows linearly with scene complexity, preserving high throughput up to 10ms/frame for 75M gaussians.

  Figure 5: HiGS exhibits linear scaling in frame time with gaussian count, maintaining best-in-class performance across scene complexities.

Numerical Robustness and Representation

HiGS exclusively utilizes half-precision (fp16) arithmetic for both storage and computation throughout the pipeline, employing Cholesky factorization for the inverse covariance matrix to stabilize the Mahalanobis quadratic form and avoid cancellation risks. All pixel and gaussian coordinates are stored relative to macro-tile centers to keep operand magnitudes well within fp16 range.

The system achieves imperceptible error relative to fp32 baselines (67 dB PSNR vs. gsplat, and 27.68 dB PSNR against ground truth), with no perceptible artifacts introduced by the half-precision pipeline. The global pipeline thus maintains both computational efficiency and numerical stability for real-time settings.

Limitations and Future Directions

The present implementation addresses forward (inference) rendering. The macro-tile decomposition is readily extensible to differentiable backward passes, with some adjustments required for batchwise gradient accumulation and transmittance handling. Extension to per-pixel sort queues, advanced antialiasing, or secondary effects (e.g., via hybrid ray tracing as in "3D Gaussian Ray Tracing") is not covered. The main hardware dependency is CUDA-enabled GPUs with sufficient shared memory to support hierarchical local storage and half-precision math.

Integration with orthogonal accelerator techniques—such as tensor-core blending, input-side pair and model reduction, or distributed and multi-GPU data-parallel schemes—is feasible and could further enhance overall performance.

Theoretical and Practical Implications

From a theoretical viewpoint, HiGS demonstrates that decoupling spatial partitioning from rasterization parallelism applies load balancing and memory traffic reduction principles successfully in explicit neural view synthesis pipelines. Practically, it brings high-throughput, exact front-to-back compositing to production-scale, multi-million gaussian reconstructions on commodity GPUs, without sacrificing image fidelity.

By eliminating the granularity trade-off inherent to the tile-based 3DGS pipeline, HiGS sets a new standard for real-time rasterization-based neural rendering architectures and presents an extensible template for hierarchical scheduling and partitioning in both forward and backward passes.

Conclusion

HiGS establishes that a two-scale spatial hierarchy, with decoupled binning and rasterization levels, yields substantial performance and scalability gains for real-time 3D Gaussian Splatting. It consistently outpaces every previously published rasterizer, achieving strong numerical fidelity, improved hardware utilization, and robust scaling with increasing scene complexity.

Potential future developments include extension to gradient-based training, integration with further compression/acceleration techniques, and exploration of hardware-aware scheduling refinements for alternative GPU architectures. The hierarchical principles underlying HiGS are transferable to other explicit representation pipelines and offer a generically superior rendering architecture for real-time neural synthesis.