Papers
Topics
Authors
Recent
Search
2000 character limit reached

High-Quality Real-Time Rendering Using Subpixel Sampling Reconstruction

Published 3 Jan 2023 in cs.CV and eess.IV | (2301.01036v2)

Abstract: Generating high-quality, realistic rendering images for real-time applications generally requires tracing a few samples-per-pixel (spp) and using deep learning-based approaches to denoise the resulting low-spp images. Existing denoising methods have yet to achieve real-time performance at high resolutions due to the physically-based sampling and network inference time costs. In this paper, we propose a novel Monte Carlo sampling strategy to accelerate the sampling process and a corresponding denoiser, subpixel sampling reconstruction (SSR), to obtain high-quality images. Extensive experiments demonstrate that our method significantly outperforms previous approaches in denoising quality and reduces overall time costs, enabling real-time rendering capabilities at 2K resolution.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Self-supervised post-correction for monte carlo denoising. In Proceedings of ACM SIGGRAPH 2022 Conference, pages 1–8, 2022.
  2. Kernel-predicting convolutional networks for denoising monte carlo renderings. ACM Transactions on Graphics (TOG), 36(4), jul 2017.
  3. Guided image filtering for interactive high-quality global illumination. In Proceedings of the Eurographics Conference on Rendering (EG), page 1361–1368, Goslar, DEU, 2011. Eurographics Association.
  4. Nonlinearly weighted first-order regression for denoising monte carlo renderings. Computer Graphics Forum (CGF), 35(4):107–117, jul 2016.
  5. Interactive reconstruction of monte carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics (TOG), 36:1–12, 07 2017.
  6. Differentiable surface rendering via non-differentiable sampling. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 6088–6097, October 2021.
  7. Machine-learning denoising in feature film production. In ACM SIGGRAPH 2019 Talks, pages 1–2. 2019.
  8. Machine-learning denoising in feature film production. In ACM SIGGRAPH 2019 Talks, SIGGRAPH ’19, New York, NY, USA, 2019. Association for Computing Machinery.
  9. Boosting monte carlo rendering by ray histogram fusion. ACM Transactions on Graphics (TOG), 33, 02 2014.
  10. Truly next-gen: Adding deep learning to games and graphics. In NVIDIA Sponsored Sessions (Game Developers Conference), 2019.
  11. Real-time monte carlo denoising with weight sharing kernel prediction network. Computer Graphics Forum (CGF), 40(4):15–27, 2021.
  12. Neural-sim: Learning to generate training data with nerf. In Proceedings of the European Conference on Computer Vision (ECCV), pages 477–493. Springer, 2022.
  13. Sample-based monte carlo denoising using a kernel-splatting network. ACM Transactions on Graphics (TOG), 38(4), jul 2019.
  14. Neural temporal adaptive sampling and denoising. In Computer Graphics Forum (CGF), volume 39, pages 147–155. Wiley Online Library, 2020.
  15. Interactive path tracing and reconstruction of sparse volumes. 4(1), apr 2021.
  16. A survey on deep learning-based monte carlo denoising. Computational visual media, 7:169–185, 2021.
  17. Interactive monte carlo denoising using affinity of neural features. ACM Transactions on Graphics (TOG), 40(4), jul 2021.
  18. James T. Kajiya. The rendering equation. In Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’86, page 143–150, New York, NY, USA, 1986. Association for Computing Machinery.
  19. A machine learning approach for filtering monte carlo noise. ACM Transactions on Graphics (TOG), 34(4), jul 2015.
  20. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, International Conference on Learning Representations (ICLR), 2015.
  21. Blockwise multi-order feature regression for real-time path-tracing reconstruction. 38(5), jun 2019.
  22. Deep adaptive sampling for low sample count rendering. Computer Graphics Forum (CGF), 37:35–44, 07 2018.
  23. Sure-based optimization for adaptive sampling and reconstruction. ACM Transactions on Graphics (TOG), 31(6), nov 2012.
  24. Phyir: Physics-based inverse rendering for panoramic indoor images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12713–12723, June 2022.
  25. Swin transformer: Hierarchical vision transformer using shifted windows. CoRR, abs/2103.14030, 2021.
  26. Amazon Lumberyard. Amazon lumberyard bistro, open research content archive (orca). http://developer.nvidia.com/orca/amazon-lumberyard-bistro. Accessed July, 2017.
  27. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proceedings of the International Conference on Neural Information Processing Systems (NeurIPS), NIPS’16, page 2810–2818, Red Hook, NY, USA, 2016. Curran Associates Inc.
  28. An efficient denoising algorithm for global illumination. In Proceedings of High Performance Graphics, HPG ’17, New York, NY, USA, 2017. Association for Computing Machinery.
  29. Real-time Monte Carlo Denoising with the Neural Bilateral Grid. In Carsten Dachsbacher and Matt Pharr, editors, Eurographics Symposium on Rendering - DL-only Track. The Eurographics Association, 2020.
  30. Adaptive rendering based on weighted local regression. ACM Transactions on Graphics (TOG), 33(5), sep 2014.
  31. Robust image denoising using a virtual flash image for monte carlo ray tracing. Computer Graphics Forum (CGF), 32(1):139–151, 2013.
  32. Adaptive polynomial rendering. ACM Transactions on Graphics (TOG), 35(4), jul 2016.
  33. Neural denoising with layer embeddings. Computer Graphics Forum (CGF), 39(4):1–12, 2020.
  34. NVIDIA. Deep learning super sampling (dlss) technology. https://www.nvidia.com/en-us/geforce/technologies/dlss/. Accessed April 4, 2021.
  35. OpenGL. Deferred shading. https://learnopengl.com /advanced-lighting/deferred-shading. . Accessed January 10, 2023.
  36. The making of welcome to light fields vr. In Proceedings of ACM SIGGRAPH 2018 Conference, pages 1–2. ACM, 2018.
  37. Pytorch: An imperative style, high-performance deep learning library. CoRR, abs/1912.01703, 2019.
  38. Gendr: A generalized differentiable renderer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4002–4011, June 2022.
  39. Mr image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Transactions on Medical Imaging, 30(5):1028–1041, 2011.
  40. U-net: Convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597, 2015.
  41. Adaptive rendering with non-local means filtering. ACM Transactions on Graphics (TOG), 31(6), nov 2012.
  42. Robust denoising using feature and color information. Computer Graphics Forum (CGF), 32(7):121–130, 2013.
  43. Spatiotemporal variance-guided filtering: Real-time reconstruction for path-traced global illumination. In Proceedings of High Performance Graphics, HPG ’17, New York, NY, USA, 2017. Association for Computing Machinery.
  44. Gradient estimation for real-time adaptive temporal filtering. Proc. ACM Comput. Graph. Interact. Tech., 1(2), aug 2018.
  45. Andrew F Seila. Simulation and the monte carlo method, 1982.
  46. Vulkan programming guide: The official guide to learning vulkan. Addison-Wesley Professional, 2016.
  47. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  48. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics (TOG), 37(4), jul 2018.
  49. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing (TIP), 13(4):600–612, 2004.
  50. Learning indoor inverse rendering with 3d spatially-varying lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12538–12547, October 2021.
  51. De-rendering 3d objects in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18490–18499, June 2022.
  52. Neural supersampling for real-time rendering. ACM Transactions on Graphics (TOG), 39(4), jul 2020.
  53. Adversarial monte carlo denoising with conditioned auxiliary feature modulation. ACM Transactions on Graphics (TOG), 38(6), nov 2019.
  54. A survey of temporal antialiasing techniques. Computer Graphics Forum (CGF), 39(2):607–621, 2020.
  55. Amortized supersampling. ACM Transactions on Graphics (TOG), 28(5):1–12, dec 2009.
  56. Monte carlo denoising via auxiliary feature guided self-attention. ACM Transactions on Graphics (TOG), 40(6), dec 2021.
  57. Recent advances in adaptive sampling and reconstruction for monte carlo rendering. Computer Graphics Forum (CGF), 34(2):667–681, may 2015.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up