Occlusion Culling MLP

• Occlusion Culling MLPs are neural network-based techniques that predict the visibility of spatial primitives, replacing traditional geometry-based culling methods.
• They leverage compact, GPU-friendly MLPs with grid encoding to accelerate rendering tasks in applications like path tracing and 3D Gaussian splatting.
• These methods improve performance by reducing computation and memory overhead, enabling faster secondary precision in ray tracing and neural rendering.

Occlusion culling MLPs are neural network-based techniques for predicting the visibility (i.e., occlusion or exposure) of spatial primitives from arbitrary viewpoints, enabling early pruning of objects or rays unlikely to contribute to the rendered image. These methods address the classical problem of irregular memory access and computation in geometry-based visibility tests—such as BVH traversal or depth-buffering—by amortizing occlusion inference into small, GPU-friendly multilayer perceptrons (MLPs). Two major application domains have emerged: accelerating secondary visibility in path tracing, and efficient occlusion-aware primitive selection in point-based neural rendering, notably 3D Gaussian Splatting. Recent approaches demonstrate that compact MLPs, trained to approximate viewpoint-dependent visibility functions, can replace geometric occlusion queries, substantially reducing computation and memory overhead while maintaining high image fidelity (Fujieda et al., 2023, Zoomers et al., 24 Nov 2025, Gao et al., 29 Sep 2025).

1. Core Principles of Occlusion Culling MLPs

Occlusion culling MLPs aim to model the binary or probabilistic function

$f: (p, d) \mapsto \{0,1\}$

that indicates whether a point or primitive at spatial location $p$ is visible from direction or camera $d$. The objective is to skip the traversal or processing of occluded elements as early as possible in the rendering pipeline, reducing the number of shading or rasterization operations.

Traditional GPU culling relies on geometric traversals (BVH, hierarchical z-buffer, proxy mesh) that suffer from divergent control flow and unpredictable memory patterns—weak points for SIMD architectures. In contrast, dense MLP inference is matrix-bound, highly parallelizable, and delivers predictable access. By encoding relevant spatial and view-dependent signals into fixed-size vectors and passing them through a trained MLP, these models output a visibility estimate for each primitive or query, which can be thresholded to produce culling decisions.

2. Network Architectures and Encoding Strategies

Neural Intersection Function (NIF) in Monte Carlo Ray Tracing

The NIF approach integrates a small, grid-conditioned MLP into a two-level BVH ray-tracing pipeline, focusing on secondary ray occlusion (Fujieda et al., 2023). Here, NIF replaces the irregular bottom-level BVH traversal with one of two MLPs (outer and inner), each conditioned via small trainable feature grids:

• Input Transformation: Rays are parameterized by their intersection/entry point on the object’s AABB and reparameterized into spherical/UV coordinates $(p', d')$. For inner rays, the normalized radial distance $r'$ is included.
• Grid Encoding: $p'$ and $d'$ (and $r'$) index into per-object trainable 2D and 1D feature grids, producing compact latent vectors which encapsulate geometric locality.
• Network Structure: The outer NIF MLP is 6→64→64→1, the inner is 13→48→48→48→1, each using Leaky-ReLU activations and no normalization. The concatenated grid features are the sole MLP inputs.
• Output: The MLP predicts a scalar visibility probability in $[0,1]$, thresholded at 0.5. All weights and grids are trainable per object.

3D Gaussian Splatting: NVGS and Proxy-GS Variants

In neural Gaussian splatting pipelines, occlusion culling MLPs are tasked with learned, view-dependent pruning of semi-transparent Gaussian primitives.

NVGS (Zoomers et al., 24 Nov 2025):

• Visibility Predictor Architecture: The central MLP receives 16-dimensional input (Gaussian local coordinates, normalized view vector, camera direction in asset frame, learned Gaussian embedding). Four hidden layers of 64 (ReLU) with a final sigmoid produce the visibility probability.
• Embedding MLP: Static Gaussian parameters are compressed to a 6-D feature with a two-layer 32-neuron MLP, precomputed per asset.
• Weight Sharing: All instances of an asset share the same weights for the central MLP; only the per-Gaussian embedding vector varies.

Proxy-GS (Gao et al., 29 Sep 2025):

• Rather than using an MLP for occlusion estimation, Proxy-GS leverages a low-complexity proxy mesh to actualize geometric visibility culling, with MLPs still used for appearance and Gaussian parameter decoding.

3. Data Generation and Training Protocols

NIF (Ray-Tracing Context) (Fujieda et al., 2023)

• Visibility Labels: Ground-truth occlusion labels are collected from secondary rays during the early sample-per-pixel (spp) phase of path tracing, leveraging existing BVH queries.
• Training Regimen:
  • Loss: Squared error loss $\mathcal{L}^2(\hat{y}, y) = \|\hat{y} - y\|^2$.
  • Optimizer: Adam with $\beta_1=0.9, \beta_2=0.999$, $\epsilon=10^{-15}$, LR=$0.005$.
  • Batch Sizes: Outer/inner MLP batch 2048/4096 rays per step.
  • Online Training: Occurs interleaved with rendering, using newly discovered rays per spp.

NVGS (3DGS Context) (Zoomers et al., 24 Nov 2025)

• Visibility Labels: Gaussians are considered visible if they contribute nonzero color to the rendering, determined via volume rendering transmittance.
• Camera/View Sampling: Cameras are distributed on a Fibonacci sphere with randomized displacement for robust coverage; auxiliary views help minimize transient effects.
• Loss and Optimization: Binary cross-entropy loss; Adam optimizer with LR starting at $2\!\times\! 10^{-3}$; batch size $2^{19}$; training runs for $\sim 30$k iterations ($\sim$1.3 min).
• Preprocessing: Gaussian embedding vectors via a secondary MLP; removal of near-transparent Gaussians.

4. Integration into Rendering Pipelines

NIF in BVH Ray-Tracing (Fujieda et al., 2023)

• Pipeline Steps:

1. Top-level BVH traversal to determine candidate (object, ray) pairs.
2. Feature grid lookup for rays hitting each object's AABB, applying appropriate outer/inner transformation.
3. MLP inference per candidate: dense, regular, GPU-optimized.
4. Only rays with predicted visible outcome continue; occluded rays update the closest hit and terminate.

• Benefits: Avoids divergent, pointer-heavy bottom-level BVH traversal. GPU occupancy and memory coherence are optimized; the cost per ray becomes independent of object triangle count.

NVGS in Instanced Rasterization (Zoomers et al., 24 Nov 2025)

• Pipeline Steps:

1. Frustum cull Gaussians in object-local coordinates.
2. For surviving Gaussians, compute the MLP input (16 D) and batch-evaluate the occlusion MLP.
3. Cull Gaussians with predicted $V(G;c) < 0.5$.
4. Standard splatting/rasterization is applied to remaining Gaussians.

• Performance: Up to 75% of Gaussians are culled pre-rasterization, yielding significant reductions in VRAM (~4×) and compute (~15–25 FPS gains).

Proxy-GS (Gao et al., 29 Sep 2025)

• Approach: A hardware rasterized proxy mesh produces a per-view depth map for occlusion culling. Anchors failing the depth test are pruned before MLP decoding and splatting. Unlike NVGS, culling is explicit geometric rather than learned via MLP.

5. Quantitative Performance and Comparative Analysis

Table: Occlusion Culling MLP Performance Excerpts

Method/Domain	PSNR↑	VRAM↓	Primitive Reduction	Speedup
NIF (Ray tracing)	31–43 dB	—	—	1.5× (35%)
NVGS (Forest, 3DGS)	52.7 dB	4.0 GB	60–75%	+15–25 FPS
Proxy-GS (MatrixCity)	22.11 dB	—	4× anchor reduction	2.5–4× FPS

NIF demonstrates up to 1.53× speedup in shadow-ray casting with negligible (<1% luminance) loss and high fidelity (31–43 dB PSNR) (Fujieda et al., 2023). NVGS achieves up to 4× VRAM reduction and a 10 dB PSNR gain over cluster-based LoD hierarchies (V3DG), and a ~25% further VRAM reduction over frustum-only approaches (Zoomers et al., 24 Nov 2025). Proxy-GS achieves a 2.5–4× FPS acceleration in occlusion-heavy scenes via geometric proxy-mesh culling followed by MLP-based decoding (Gao et al., 29 Sep 2025).

6. Interaction with Frustum Culling and Level-of-Detail

Occlusion culling MLPs are orthogonal to conventional frustum culling, which eliminates off-screen primitives. They are typically applied after frustum culling but before shading/rasterization to reject primitives occluded in current view. Integration with LoD hierarchies (e.g., V3DG) remains complementary; after MLP-based occlusion culling, surviving primitives can be further replaced or merged by LoD strategies for far distances (Zoomers et al., 24 Nov 2025).

In 3DGS pipelines, this multi-stage culling (frustum → MLP occlusion → LoD) enables both superior frame rates and minimized memory load, as observed in NVGS and Proxy-GS.

7. Limitations, Quality, and Scalability Considerations

• Image Quality: Errors introduced by MLP-based occlusion culling are stochastic and diminish with higher grid/MLP capacity and more training samples. PSNRs of 30–60 dB can be achieved, with most errors confined to thin shadow regions or highly occluded points (Fujieda et al., 2023, Zoomers et al., 24 Nov 2025).
• Bias: No systematic bias is observed; errors are reduced through extended training or higher grid/MLP resolutions.
• Scalability: MLP inference cost is decoupled from scene or primitive complexity, making these approaches amenable to scenes with millions of triangles or Gaussians.
• Implementation: Constant memory access patterns and dense MLP computation map effectively onto hardware-accelerated (e.g., WMMA/TensorCore) GPU instructions, minimizing thread divergence and maximizing throughput.
• Limitations: A plausible implication is that for extremely non-local occlusion phenomena or if the visibility boundary is non-smooth, MLP-based approximations may require larger architectures or increased training effort.

• Neural Intersection Function (Fujieda et al., 2023)
• NVGS: Neural Visibility for Occlusion Culling in 3D Gaussian Splatting (Zoomers et al., 24 Nov 2025)
• Proxy-GS: Efficient 3D Gaussian Splatting via Proxy Mesh (Gao et al., 29 Sep 2025)