CoherentRaster: Efficient 3D Gaussian Splatting for Light Field Displays

Abstract: Light field displays (LFDs) require rendering an interlaced image that encodes many view-dependent observations. This multi-view requirement introduces substantial computational overhead, making real-time rendering difficult to achieve. While 3D Gaussian Splatting (3DGS) is efficient for single-view rendering on 2D displays, directly extending it to LFDs is computationally expensive. Moreover, prior accelerations either suffer from GPU inefficiency under spatially incoherent subpixel layouts or rely on computationally heavy multi-plane intermediates. In this paper, we propose CoherentRaster, a 3DGS-based light field rendering framework that performs subpixel-level rasterization. Our method employs Cross-view Coherent Attribute Reuse to eliminate redundant computation across neighboring viewpoints and applies View-coherent Remapping to restore warp-level memory efficiency degraded by the interlaced subpixel layout. Together, CoherentRaster provides an efficient pipeline for real-time, high-quality light field synthesis on consumer-grade hardware.

• The paper introduces CoherentRaster, which extends 3D Gaussian Splatting to light field displays using cross-view coherent attribute reuse and view-coherent remapping.
• It clusters neighboring viewpoints to eliminate redundant per-subpixel computations, achieving impressive real-time performance (e.g., 88 FPS at 2K on synthetic scenes).
• Extensive evaluations confirm that the method maintains visual fidelity while drastically improving FPS compared to baseline approaches, enabling interactive 3D LFD applications.

CoherentRaster: Efficient 3D Gaussian Splatting for Light Field Displays

Introduction

"CoherentRaster: Efficient 3D Gaussian Splatting for Light Field Displays" (2605.04509) addresses the computational bottlenecks inherent in rendering for light field displays (LFDs) using 3D Gaussian Splatting (3DGS). Although 3DGS achieves real-time, high-quality single-view synthesis on 2D displays, direct extension to LFDs is impractical due to the exponential multi-view computation required for generating interlaced, viewpoint-dependent images. Prior approaches—such as naive multi-view rasterization and multi-plane image (MPI) intermediates—either fail to exploit inter-view coherence or incur excessive memory and computational overhead, especially at high resolutions. This paper proposes a highly optimized rasterization framework, CoherentRaster, that incorporates both algorithmic and architectural strategies to achieve real-time, high-fidelity LFD rendering on commodity GPUs.

Figure 1: Principle of lenticular LFDs, illustrating the viewpoint index matrix and mapping between subpixels and viewpoints.

Light Field Display Rasterization Challenges

LFDs, especially lenticular variants, require each subpixel to encode the color information as seen from a unique viewpoint, as dictated by a display-specific viewpoint index matrix. This arrangement shatters the spatial locality assumption underlying GPU efficiency, since adjacent subpixels often correspond to disparate views. Additionally, interlacing the output from tens to hundreds of views at high spatial resolutions (2K–4K) exacerbates the throughput demand.

Traditional approaches that render individual images for all viewpoints, then interlace, are prohibitively expensive, while pure subpixel-level renderers compromise hardware efficiency via memory divergence. MPI strategies share computation across views, but scalability suffers due to required plane counts for artifact-free results.

CoherentRaster: Pipeline and Optimization Strategies

CoherentRaster builds upon the 3DGS paradigm but adapts its rasterization architecture for LFD requirements by two principal mechanisms: Cross-view Coherent Attribute Reuse and View-coherent Remapping. The pipeline operates directly at subpixel granularity, determining for each subpixel the set of contributing Gaussians and compositing their attributes efficiently.

Figure 2: CoherentRaster pipeline leveraging Cross-view Coherent Attribute Reuse and View-coherent Remapping for optimized high-resolution light field rendering.

Subpixel-Level Rasterization

For each subpixel $(x, y, u)$, CoherentRaster uses the viewpoint index matrix to identify the responsible viewpoint and only computes the corresponding Gaussian contributions. This avoids much redundant work characteristic of naive approaches.

Cross-view Coherent Attribute Reuse

Recognizing that Gaussian attributes such as covariance, depth, and color (from SH) change smoothly with small viewpoint shifts, CoherentRaster clusters neighboring viewpoints and reuses computed attributes within each cluster. Only the 2D projected mean is computed per view to maintain geometric accuracy, while more stable attributes are computed and shared cluster-wise. This drastically reduces the number of Gaussian-tile pairs and resultant sorting workload, improving both run-time and memory utilization.

View-coherent Remapping

The mapping between threads and subpixels is reordered such that threads within each warp process subpixels associated with the same or spatially-close viewpoints. This alignment nearly restores memory coalescing and warp efficiency, which are lost with naive subpixel-level mapping. The lookup table for this mapping is precomputed based on fixed display optics.

Quantitative and Qualitative Evaluation

Extensive experiments validate the efficacy of CoherentRaster on both synthetic (Blender) and real-world (Mip-NeRF 360) 3D scenes using commercial LFDs (2K and 4K). Compared to the baseline 3DGS pipeline and contemporary MPI or subpixel-based methods, CoherentRaster attains substantial FPS improvements with negligible loss in visual quality.

• On high-end GPUs (RTX 5090), CoherentRaster delivers 88 FPS at 2K and 56 FPS at 4K on synthetic scenes (compared to 5.8 and 4.1 FPS, respectively, for 3DGS).
• Real-world, high-complexity scenes sustain 30 FPS at 2K and 16 FPS at 4K, enabling interactive applications.
• PSNR, SSIM, and LPIPS scores remain tightly coupled to the 3DGS pseudo-ground truth across all cluster sizes considered.
• MPI-based methods suffer from severe discretization artifacts and lower PSNR, even as their computational cost increases with depth plane count.

  Figure 3: Qualitative comparison—CoherentRaster maintains perceptual quality nearly indistinguishable from full-frame 3DGS on diverse LFD scenes.

Component Analysis and Limitations

Isolated ablation of Cross-view Coherent Attribute Reuse and View-coherent Remapping demonstrates their synergistic contributions: neither alone suffices for real-time on complex scenes, but together they yield maximal throughput. When varying cluster size, visual quality remains stable for moderate cluster sizes, while excessive clustering can introduce mild artifacts.

Figure 4: Visual quality is maintained across a range of viewpoint cluster sizes; large clusters risk minor distortion.

Limitations are observed on highly specular surfaces, where Cross-view Coherent Attribute Reuse can leak high-frequency view-dependent effects, producing visible artifacts. This is mitigated in practice by dense angular sampling and the smoothing effect of low-order SH.

Figure 5: Artifacts appear on highly specular surfaces due to inter-view attribute sharing.

Practical and Theoretical Implications

Practically, CoherentRaster represents an enabling technology for interactive, artifact-free 3D visualization on autostereoscopic LFDs using commodity hardware, without the bandwidth or memory bottlenecks of prior dense radiance field renderers. The approach is extensible to any GPU, including those with small on-chip caches, due to reduced memory divergence. Architecturally, it exposes a general pattern for addressing hardware inefficiency in multi-view rasterization pipelines, balancing per-view accuracy with attribute sharing while leveraging spatial coherence for optimal warps.

Theoretically, this work suggests that clustering-based attribute reuse can be systematized for other multi-view or view-dependent rendering problems. Extending CoherentRaster to real-time dynamic scene synthesis, or to spatial light modulators with different optical layouts, is a natural future direction. Hybrid models may also adapt cluster granularity based on scene content, e.g., adaptive clustering for high-frequency regions.

Conclusion

CoherentRaster introduces a scalable, real-time pipeline for 3DGS-based LFD rendering, combining subpixel-level work elimination with hardware-aware remapping and inter-view attribute reuse. The framework achieves significant speedups over prior art without compromising visual fidelity, making it the current method of choice for interactive 3D light field display applications. Future research should explore adaptive clustering, optimization for dynamic content, and integration with advanced radiance field representations to further boost efficiency and generality.