Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Sort-free Gaussian Splatting via Weighted Sum Rendering (2410.18931v1)

Published 24 Oct 2024 in cs.CV

Abstract: Recently, 3D Gaussian Splatting (3DGS) has emerged as a significant advancement in 3D scene reconstruction, attracting considerable attention due to its ability to recover high-fidelity details while maintaining low complexity. Despite the promising results achieved by 3DGS, its rendering performance is constrained by its dependence on costly non-commutative alpha-blending operations. These operations mandate complex view dependent sorting operations that introduce computational overhead, especially on the resource-constrained platforms such as mobile phones. In this paper, we propose Weighted Sum Rendering, which approximates alpha blending with weighted sums, thereby removing the need for sorting. This simplifies implementation, delivers superior performance, and eliminates the "popping" artifacts caused by sorting. Experimental results show that optimizing a generalized Gaussian splatting formulation to the new differentiable rendering yields competitive image quality. The method was implemented and tested in a mobile device GPU, achieving on average $1.23\times$ faster rendering.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (56)
  1. Neural point-based graphics. In European Conference on Computer Vision, pp.  696–712. Springer, 2020.
  2. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5470–5479, 2022.
  3. Multi-fragment effects on the gpu using the k-buffer. In Proceedings of the 2007 symposium on Interactive 3D graphics and games, pp.  97–104, 2007.
  4. Revising densification in gaussian splatting. arXiv preprint arXiv:2404.06109, 2024.
  5. Loren Carpenter. The a-buffer, an antialiased hidden surface method. In Proceedings of the 11th annual conference on Computer graphics and interactive techniques, pp.  103–108, 1984.
  6. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision (ECCV), 2022.
  7. A survey on 3d gaussian splatting. arXiv preprint arXiv:2401.03890, 2024a.
  8. A Survey on 3D Gaussian Splatting, July 2024b. URL http://arxiv.org/abs/2401.03890. arXiv:2401.03890 [cs].
  9. Hac: Hash-grid assisted context for 3d gaussian splatting compression. arXiv preprint arXiv:2403.14530, 2024.
  10. A neural rendering framework for free-viewpoint relighting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5599–5610, 2020.
  11. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In The Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
  12. Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision, pp.  764–773, 2017.
  13. Stochastic transparency. In Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, pp.  157–164, 2010.
  14. Cass Everitt. Interactive order-independent transparency. White paper, nVIDIA, 2(6):7, 2001.
  15. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps, 2023.
  16. Instantsplat: Unbounded sparse-view pose-free gaussian splatting in 40 seconds. arXiv preprint arXiv:2403.20309, 2024.
  17. 3D Gaussian as a New Era: A Survey. IEEE Transactions on Visualization and Computer Graphics, pp.  1–20, 2024. ISSN 1077-2626, 1941-0506, 2160-9306. doi: 10.1109/TVCG.2024.3397828. URL http://arxiv.org/abs/2402.07181. arXiv:2402.07181 [cs].
  18. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.  5501–5510, 2022.
  19. Colmap-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  20796–20805, June 2024.
  20. Ges: Generalized exponential splatting for efficient radiance field rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  19812–19822, 2024.
  21. Deep blending for free-viewpoint image-based rendering. 37(6):257:1–257:15, 2018.
  22. Fisherrf: Active view selection and uncertainty quantification for radiance fields using fisher information. arXiv preprint arXiv:2311.17874, 2023.
  23. Identifying unnecessary 3d gaussians using clustering for fast rendering of 3d gaussian splatting. arXiv preprint arXiv:2402.13827, 2024.
  24. 3D Gaussian Splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023.
  25. kishimisu. kishimisu/Gaussian-Splatting-WebGL, September 2024. URL https://github.com/kishimisu/Gaussian-Splatting-WebGL. original-date: 2023-10-19T19:39:24Z.
  26. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1–13, 2017.
  27. Kevin Kwok. antimatter15/splat, September 2024. URL https://github.com/antimatter15/splat. original-date: 2023-09-11T02:24:56Z.
  28. Compact 3d gaussian representation for radiance field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  21719–21728, 2024.
  29. Spacetime gaussian feature splatting for real-time dynamic view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  8508–8520, June 2024.
  30. Neulf: Efficient novel view synthesis with neural 4d light field. In Eurographics Symposium on Rendering, 2022. URL https://api.semanticscholar.org/CorpusID:236913029.
  31. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  21644–21653, 2024.
  32. Vastgaussian: Vast 3d gaussians for large scene reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  5166–5175, 2024.
  33. Neural volumes: Learning dynamic renderable volumes from images. ACM Trans. Graph., 38(4):65:1–65:14, July 2019.
  34. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  20654–20664, 2024.
  35. Fundamentals of Computer Graphics. A K Peters, Wellesley, MA, fourth edition, 2015. ISBN 978-1482229394.
  36. Weighted Blended Order-Independent Transparency. Journal of Computer Graphics Techniques, 2(4), 2013.
  37. Houman Meshkin. Sort-Independent Alpha Blending, 2007.
  38. Nerf: representing scenes as neural radiance fields for view synthesis. Commun. ACM, 65(1):99–106, dec 2021. ISSN 0001-0782. doi: 10.1145/3503250. URL https://doi.org/10.1145/3503250.
  39. Compact 3d scene representation via self-organizing gaussian grids. arXiv preprint arXiv:2312.13299, 2023.
  40. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, July 2022. doi: 10.1145/3528223.3530127. URL https://doi.org/10.1145/3528223.3530127.
  41. Compact3d: Compressing gaussian splat radiance field models with vector quantization. arXiv preprint arXiv:2311.18159, 2023.
  42. Donerf: Towards real-time rendering of compact neural radiance fields using depth oracle networks. In Computer Graphics Forum, volume 40, pp.  45–59. Wiley Online Library, 2021.
  43. Compressed 3d gaussian splatting for accelerated novel view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.  10349–10358, June 2024.
  44. NVIDIA. CUB :: CUDA Toolkit Documentation, 2024. URL https://docs.nvidia.com/cuda/cub/index.html#cub-overview. Accessed: 2024-10-23.
  45. Compositing digital images. In Proceedings of the 11th annual conference on Computer graphics and interactive techniques, pp.  253–259, 1984.
  46. Langsplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  20051–20060, 2024.
  47. Stopthepop: Sorted gaussian splatting for view-consistent real-time rendering. ACM Trans. Graph., 43(4), jul 2024. ISSN 0730-0301. doi: 10.1145/3658187. URL https://doi.org/10.1145/3658187.
  48. Deepvoxels: Learning persistent 3d feature embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.  2437–2446, 2019.
  49. Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics (TOG), 38(4):1–12, 2019.
  50. Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pp.  6411–6420, 2019.
  51. Recent advances in 3d gaussian splatting. Computational Visual Media, pp.  1–30, 2024.
  52. Chris Wyman. Exploring and expanding the continuum of oit algorithms. In High Performance Graphics, pp.  1–11, 2016.
  53. Multi-scale 3d gaussian splatting for anti-aliased rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  20923–20931, 2024.
  54. Mip-splatting: Alias-free 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  19447–19456, 2024.
  55. Fregs: 3d gaussian splatting with progressive frequency regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  21424–21433, 2024.
  56. Triplane meets gaussian splatting: Fast and generalizable single-view 3d reconstruction with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10324–10335, 2024.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Qiqi Hou (11 papers)
  2. Randall Rauwendaal (2 papers)
  3. Zifeng Li (5 papers)
  4. Hoang Le (14 papers)
  5. Farzad Farhadzadeh (7 papers)
  6. Fatih Porikli (141 papers)
  7. Alexei Bourd (1 paper)
  8. Amir Said (18 papers)
Citations (1)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets