Proxy-GS: Occlusion-Aware MLP 3D Gaussian Splatting

• Proxy-GS is an occlusion-aware framework for MLP-based 3D Gaussian Splatting that uses a coarse proxy mesh to generate per-view depth maps for effective visibility culling.
• It reduces computational overhead by culling occluded anchors before MLP decoding, achieving speedups and enhanced image quality in heavily occluded settings.
• The method leverages proxy-guided densification to place new anchors on proxy surfaces, ensuring geometric consistency and efficient anchor management.

Proxy-GS denotes an occlusion-aware training and inference framework for MLP-based 3D Gaussian Splatting (3DGS) built around a lightweight proxy mesh and hardware rasterization. In its canonical formulation, it is designed for large, heavily occluded scenes, particularly anchor-based and level-of-detail (LOD) pipelines such as Octree-GS, where substantial redundancy remains because occlusion is not modeled explicitly during anchor decoding and densification (Gao et al., 29 Sep 2025). In the broader 3DGS literature, the term “proxy” is also used for cell proxies in order-independent rendering and for mesh proxies in deformation systems; accordingly, Proxy-GS sits within a wider family of proxy-mediated Gaussian methods, but its defining contribution is the use of a coarse mesh to produce per-view depth maps that guide both visibility culling and surface-aligned anchor growth (Gao et al., 29 Sep 2025).

1. Conceptual basis and redundancy model

Proxy-GS assumes the MLP-based 3DGS setting in which a scene is represented by anchors, each carrying a latent feature vector and decoded by an MLP into Gaussian primitives conditioned on the current view. The paper expresses this as

$$\{\mu_j,\Sigma_j,c_j,\alpha_j\}_{j\in\mathcal{M}} = \mathrm{MLP}_\theta(f_i, v_i)_{i\in\mathcal{N}},$$

where $f_i$ is the latent feature for anchor $i$, $v_i$ is the view direction, and $\mathcal{N}$ denotes the anchors selected for the current view (Gao et al., 29 Sep 2025). This formulation inherits the efficiency benefits of anchor+MLP representations such as Scaffold-GS and Octree-GS, but it also inherits their central inefficiency: distance-based LOD can remove far anchors, yet it does not remove anchors that are fully occluded by foreground geometry at comparable depth.

The motivating observation is that, in large urban or indoor scenes, many anchors decoded by Octree-GS are completely occluded from the current camera. These anchors still incur MLP decode cost and can still emit Gaussians that are immediately over-occluded in compositing. Proxy-GS identifies this lack of occlusion awareness as the major remaining redundancy after LOD and basic pruning. The same issue appears during densification: when densification is driven only by image reprojection error or gradient, new anchors may be spawned behind front surfaces, inside buildings, or in regions that are never jointly visible. This produces additional decoding overhead and can degrade geometric consistency (Gao et al., 29 Sep 2025).

The framework is therefore organized around two linked claims. First, a coarse geometric prior is sufficient to estimate view-dependent first-surface depth at very low cost. Second, if that depth is injected directly into both training and inference, the anchor set can be made visibility-aware before MLP decoding. This suggests a shift in where acceleration is applied: rather than compressing only the Gaussian set after decoding, Proxy-GS reduces the number of anchors that invoke the decoder at all.

2. Proxy mesh subsystem and depth-map generation

The proxy in Proxy-GS is a lightweight proxy mesh that approximates scene geometry and is used purely as a geometric prior; it does not carry radiance or texture, and it remains fixed with respect to optimization (Gao et al., 29 Sep 2025). The reconstruction route depends on available data. If dense point clouds or depth are available, the mesh can be obtained by depth fusion into TSDF or by surface reconstruction such as Neural Kernel Surface Reconstruction. For indoor scenes with only sparse COLMAP points, the paper describes combining COLMAP structure with monocular depth estimation by MoGe-2 and multi-view refinement by PGSR. For outdoor sparse COLMAP, it cites CityGS-X as a large-scale surface reconstruction pipeline. After reconstruction, the mesh is simplified with QEM to retain coarse structures such as buildings, walls, and floors while reducing triangle count (Gao et al., 29 Sep 2025).

For each camera view, the proxy mesh is split into clusters with precomputed AABBs and subjected to frustum culling and hierarchical Z-buffer culling. A depth-only pass is then executed using the GPU fixed-function rasterizer and depth test, with minimal fragment-shader work, Early-Z, and Hi-Z enabled. The resulting depth map is produced at resolution $1000\times 1000$ in under $1$ ms on consumer GPUs, and remains on GPU through Vulkan–CUDA interop, avoiding CPU readback (Gao et al., 29 Sep 2025).

This depth pass is not a differentiable component of the training objective. It is a fixed visibility filter. Its role is to expose the depth of the first proxy surface along each camera ray at sufficiently low overhead that it can be used per batch during training and per frame during inference. A plausible implication is that Proxy-GS treats the rasterizer not as a final renderer, but as an auxiliary geometric oracle embedded into the Gaussian pipeline.

3. Occlusion-aware culling before MLP decoding

The central operational mechanism in Proxy-GS is anchor culling in screen space using the proxy depth map. For an anchor at world position $\mathbf{p}_{\text{orig}}=(x,y,z)$, the framework transforms it to view space and clip space,

$$\mathbf{p}_{\text{view}} = V \begin{bmatrix} \mathbf{p}_{\text{orig}}\ 1 \end{bmatrix},\qquad \mathbf{p}_{\text{hom}} = P\,\mathbf{p}_{\text{view}} = (x_h,y_h,z_h,w_h)^\top,$$

then converts to normalized device coordinates

$$\mathbf{p}_{\text{ndc}} = \left( \frac{x_h}{w_h+\epsilon}, \frac{y_h}{w_h+\epsilon}, \frac{z_h}{w_h+\epsilon} \right), \qquad \epsilon = 10^{-7}.$$

Anchors with $f_i$0, where $f_i$1, are discarded, and the surviving anchors are projected to image coordinates $f_i$2 (Gao et al., 29 Sep 2025).

The proxy depth at that pixel, $f_i$3, is converted to linear camera-space depth as

$f_i$4

with near and far plane distances $f_i$5. Proxy-GS then applies a safety margin $f_i$6,

$f_i$7

and culls the anchor if

$f_i$8

In effect, if an anchor lies behind the first proxy surface along the ray by more than the margin, it is treated as occluded and its Gaussians are never decoded (Gao et al., 29 Sep 2025).

This test is fused with standard frustum culling in a single CUDA kernel. Only the visible, frustum-clipped anchors are passed downstream to the Octree-GS-style MLP decoder and the 3DGS renderer. On MatrixCity Block 5, the reported average number of decoded anchors per frame drops from approximately $f_i$9 in Octree-GS to $i$0–$i$1 in Proxy-GS while image quality slightly improves (Gao et al., 29 Sep 2025). The safety margin $i$2 controls the aggressiveness of culling; on Small City, $i$3 gives PSNR $i$4 and $i$5 FPS, while $i$6 introduces artifacts and $i$7 retains more anchors with a slight slowdown (Gao et al., 29 Sep 2025).

4. Surface-guided densification and training pipeline

Proxy-GS also modifies densification so that new anchors are placed on proxy surfaces rather than in arbitrary local 3D neighborhoods. For each training image, pixels are partitioned into patches $i$8, with patch loss

$i$9

and frame-average patch loss

$v_i$0

Hard patches satisfy

$v_i$1

For each selected patch, Proxy-GS takes a representative pixel, reads hardware depth, converts it to linear depth, and back-projects to a 3D world-space point

$v_i$2

The new anchor is then placed directly at $v_i$3 (Gao et al., 29 Sep 2025).

To control local redundancy, the framework overlays a 3D proxy grid with cell size $v_i$4 and origin $v_i$5. An anchor at $v_i$6 is assigned to grid cell

$v_i$7

and is inserted only if the occupancy counter for that cell is below a cap $v_i$8 (Gao et al., 29 Sep 2025). This enforces bounded anchor density per volume and biases growth toward under-represented surface regions.

The rest of the representation remains that of Octree-GS: anchors organized in an octree with multiple LOD levels, learned feature vectors $v_i$9, and an MLP decoder that outputs Gaussian parameters for visible anchors only. The training objective remains photometric reconstruction, and the proxy is not differentiated through. Training proceeds by repeatedly rasterizing the proxy mesh to depth, applying frustum and occlusion culling, decoding the remaining anchors, rendering with a chosen 3DGS kernel, and periodically inserting new anchors through proxy-guided densification. The final experimental configuration trains for $\mathcal{N}$0 iterations (Gao et al., 29 Sep 2025).

5. Empirical behavior and operating regimes

Proxy-GS is reported to be most advantageous in heavily occluded scenes. On MatrixCity Streets, averaged over Blocks 1–2, Octree-GS obtains PSNR $\mathcal{N}$1, SSIM $\mathcal{N}$2, LPIPS $\mathcal{N}$3, and $\mathcal{N}$4 FPS, whereas Proxy-GS obtains PSNR $\mathcal{N}$5, SSIM $\mathcal{N}$6, LPIPS $\mathcal{N}$7, and $\mathcal{N}$8 FPS. On Blocks 3–4, the corresponding numbers are $\mathcal{N}$9 for Octree-GS and $1000\times 1000$0 for Proxy-GS. On Block 5, they are $1000\times 1000$1 and $1000\times 1000$2, respectively (Gao et al., 29 Sep 2025). These results amount to approximately $1000\times 1000$3–$1000\times 1000$4 speedups over Octree-GS with improved image metrics.

The comparison to vanilla 3DGS is also scene-dependent. On MatrixCity Block 1–2, 3DGS achieves PSNR $1000\times 1000$5, SSIM $1000\times 1000$6, LPIPS $1000\times 1000$7, and $1000\times 1000$8 FPS, while Proxy-GS reaches $1000\times 1000$9 (Gao et al., 29 Sep 2025). The advantage is thus not merely in acceleration of MLP-based Gaussian pipelines; in highly occluded street blocks, the visibility-aware anchor budget can make Proxy-GS faster than original 3DGS while also improving quality.

In less occluded scenes, the gains are smaller but still positive. On CUHK-LOWER, Proxy-GS improves Octree-GS from $1$0 to $1$1. On Berlin, it improves $1$2 to $1$3. On Small City, which is more occlusion-rich, the change is larger, from $1$4 to $1$5 (Gao et al., 29 Sep 2025). The decoded-anchor statistics track this pattern: for example, Block 1–2 drops from $1$6 to $1$7, Block 3–4 from $1$8 to $1$9, Block 5 from $\mathbf{p}_{\text{orig}}=(x,y,z)$0 to $\mathbf{p}_{\text{orig}}=(x,y,z)$1, and Small City from $\mathbf{p}_{\text{orig}}=(x,y,z)$2 to $\mathbf{p}_{\text{orig}}=(x,y,z)$3 (Gao et al., 29 Sep 2025).

The method is renderer-complementary rather than renderer-specific. With Proxy-GS combined with the original 3DGS renderer, the paper reports PSNR $\mathbf{p}_{\text{orig}}=(x,y,z)$4 and $\mathbf{p}_{\text{orig}}=(x,y,z)$5 FPS; with FlashGS, the same quality and $\mathbf{p}_{\text{orig}}=(x,y,z)$6 FPS; with hardware 3DGS, PSNR $\mathbf{p}_{\text{orig}}=(x,y,z)$7 and $\mathbf{p}_{\text{orig}}=(x,y,z)$8 FPS (Gao et al., 29 Sep 2025). This indicates that proxy-guided visibility pruning operates upstream of the particular Gaussian rasterization backend.

6. Related proxy paradigms in Gaussian splatting

The term “proxy” in Gaussian splatting does not refer to a single architectural pattern. Proxy-GS is specifically mesh-guided occlusion-aware training and inference, but adjacent work uses proxies in substantially different ways.

Work	Proxy type	Primary role
"Proxy-GS: Efficient 3D Gaussian Splatting via Proxy Mesh" (Gao et al., 29 Sep 2025)	Lightweight proxy mesh	Depth maps for occlusion culling and densification
"Duplex-GS: Proxy-Guided Weighted Blending for Real-Time Order-Independent Gaussian Splatting" (Liu et al., 5 Aug 2025)	Cells	Cell-level sorting, weighted blending, early termination
"UniMGS: Unifying Mesh and 3D Gaussian Splatting with Single-Pass Rasterization and Proxy-Based Deformation" (Xiao et al., 27 Jan 2026)	Proxy mesh	Gaussian-centric deformation binding and unified mesh+3DGS rasterization
"Proxy-Free Gaussian Splats Deformation with Splat-Based Surface Estimation" (Kim et al., 24 Nov 2025)	No external proxy	Surface-aware splat graph replacing mesh or cage proxies

In Duplex-GS, the proxy is not a mesh but a sparse set of ellipsoidal cells, each with a local latent feature and multiple internal Gaussians. The cells act as a proxy layer for local Gaussian management, enabling cell search rasterization, cell-level radix sort, and a weighted sum rendering formulation with cell-level transmittance $\mathbf{p}_{\text{orig}}=(x,y,z)$9 and early termination (Liu et al., 5 Aug 2025). This is a rendering and sorting proxy, not an occlusion-depth proxy.

In UniMGS, a proxy mesh serves yet another role. There, the mesh is a deformation scaffold for already trained 3DGS, with Gaussians bound to mesh faces after the fact by ray casting from training cameras, optionally via Gaussian bounding-box corners. The same work also introduces a single-pass rasterization framework that sorts mesh and Gaussian fragments together and blends them by anti-aliased $$\mathbf{p}_{\text{view}} = V \begin{bmatrix} \mathbf{p}_{\text{orig}}\ 1 \end{bmatrix},\qquad \mathbf{p}_{\text{hom}} = P\,\mathbf{p}_{\text{view}} = (x_h,y_h,z_h,w_h)^\top,$$0-blending, addressing nested occlusions and transparency between triangles and Gaussian splats (Xiao et al., 27 Jan 2026). Proxy-GS, by contrast, does not attempt unified mesh+Gaussian rendering; its mesh exists only to provide visibility and densification guidance.

SpLap explicitly frames itself as a proxy-free alternative to mesh- and cage-based Gaussian deformation. Instead of reconstructing a mesh proxy, it builds a surface-aware splat graph based on intersection of elliptical splat supports and derives a Laplacian operator directly on splats, enabling ARAP and BBW deformation without external proxy geometry (Kim et al., 24 Nov 2025). This marks the opposite pole of the proxy design space: whereas Proxy-GS inserts a coarse mesh to improve efficiency in large static scenes, SpLap removes the mesh entirely to avoid proxy-quality dependence in deformation.

7. Limitations and research directions

Proxy-GS inherits the strengths and weaknesses of coarse geometric priors. Its effectiveness depends on a reasonably good proxy mesh. Very poor meshes, such as missing large walls or misaligned floors, can cause over-aggressive culling with holes and missing content, or under-culling with less speedup. Thin and complex structures absent in the proxy may also never guide densification properly, although the framework notes that small errors can be mitigated because densified anchors are still trained by photometric loss (Gao et al., 29 Sep 2025).

The formulation also assumes a static proxy. Dynamic objects and non-rigid scenes are not handled by a single fixed proxy mesh. Extending the approach to dynamic scenes would require time-varying proxies or local proxies for moving objects. The paper further identifies extra preprocessing as a practical cost, since offline mesh reconstruction may itself be non-trivial, and notes that larger scenes may stress memory because 3DGS already faces memory and runtime issues at very large scale (Gao et al., 29 Sep 2025).

The forward-looking directions identified around Proxy-GS are correspondingly geometric and systems-oriented: better or learned proxies, integration with more advanced LOD or caching structures such as GS-Cache and LODGE, coupling with SLAM or online mapping to supply proxies in real time, and dynamic-scene extensions that combine proxy-guided visibility reasoning with 4D Gaussian Splatting (Gao et al., 29 Sep 2025). More broadly, the surrounding literature suggests two diverging trajectories. One is toward richer proxies, as in cell-based order-independent renderers and mesh-bound deformation systems (Liu et al., 5 Aug 2025, Xiao et al., 27 Jan 2026). The other is toward eliminating the proxy once the Gaussian representation itself becomes sufficiently structured, as in splat-graph deformation (Kim et al., 24 Nov 2025). Proxy-GS occupies the middle ground: it preserves standard MLP-based Gaussian representations, but introduces just enough geometry to make visibility and densification explicitly occlusion-aware.