Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FROST-BRDF: A Fast and Robust Optimal Sampling Technique for BRDF Acquisition (2401.07283v1)

Published 14 Jan 2024 in cs.GR and cs.CV

Abstract: Efficient and accurate BRDF acquisition of real world materials is a challenging research problem that requires sampling millions of incident light and viewing directions. To accelerate the acquisition process, one needs to find a minimal set of sampling directions such that the recovery of the full BRDF is accurate and robust given such samples. In this paper, we formulate BRDF acquisition as a compressed sensing problem, where the sensing operator is one that performs sub-sampling of the BRDF signal according to a set of optimal sample directions. To solve this problem, we propose the Fast and Robust Optimal Sampling Technique (FROST) for designing a provably optimal sub-sampling operator that places light-view samples such that the recovery error is minimized. FROST casts the problem of designing an optimal sub-sampling operator for compressed sensing into a sparse representation formulation under the Multiple Measurement Vector (MMV) signal model. The proposed reformulation is exact, i.e. without any approximations, hence it converts an intractable combinatorial problem into one that can be solved with standard optimization techniques. As a result, FROST is accompanied by strong theoretical guarantees from the field of compressed sensing. We perform a thorough analysis of FROST-BRDF using a 10-fold cross-validation with publicly available BRDF datasets and show significant advantages compared to the state-of-the-art with respect to reconstruction quality. Finally, FROST is simple, both conceptually and in terms of implementation, it produces consistent results at each run, and it is at least two orders of magnitude faster than the prior art.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. G. J. Ward, “Measuring and modeling anisotropic reflection,” in Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’92.   New York, NY, USA: ACM, 1992, pp. 265––272.
  2. K. J. Dana, B. van Ginneken, S. K. Nayar, and J. J. Koenderink, “Reflectance and texture of real-world surfaces,” ACM Trans. Graph., vol. 18, no. 1, pp. 1––34, jan 1999.
  3. W. Matusik, H. Pfister, M. Brand, and L. McMillan, “A data-driven reflectance model,” ACM Trans. Graph., vol. 22, no. 3, pp. 759––769, jul 2003.
  4. H. P. A. Lensch, M. Goesele, Y.-Y. Chuang, T. Hawkins, S. Marschner, W. Matusik, and G. Mueller, “Realistic materials in computer graphics,” in SIGGRAPH 2005 Courses, ser. SIGGRAPH ’05.   NY, USA: ACM, 2005.
  5. A. Ghosh, W. Heidrich, S. Achutha, and M. O’Toole, “A Basis Illumination Approach to BRDF Measurement,” International Journal on Computer Vision, vol. 90, no. 2, pp. 183–197, 2010.
  6. X. Zhou and N. K. Kalantari, “Adversarial single-image svbrdf estimation with hybrid training,” Computer Graphics Forum, vol. 40, no. 2, pp. 315–325, 2021.
  7. L. Shi, B. Li, M. Hašan, K. Sunkavalli, T. Boubekeur, R. Mech, and W. Matusik, “Match: Differentiable material graphs for procedural material capture,” ACM Trans. Graph., vol. 39, no. 6, nov 2020.
  8. V. Deschaintre, M. Aittala, F. Durand, G. Drettakis, and A. Bousseau, “Single-image svbrdf capture with a rendering-aware deep network,” ACM Transactions on Graphics (ToG), vol. 37, no. 4, pp. 1–15, 2018.
  9. S. Rusinkiewicz, “A new change of variables for efficient brdf representation,” in Rendering Techniques, ser. Eurographics.   Springer, 1998, pp. 11–22.
  10. W. Matusik, H. Pfister, M. Brand, and L. McMillan, “Efficient isotropic brdf measurement,” in Proceedings of the 14th Eurographics Workshop on Rendering, ser. EGRW ’03.   Goslar, DEU: Eurographics Association, 2003, pp. 241––247.
  11. J. Dupuy and W. Jakob, “An adaptive parameterization for efficient material acquisition and rendering,” ACM Trans. Graph., vol. 37, no. 6, dec 2018.
  12. J. B. Nielsen, H. W. Jensen, and R. Ramamoorthi, “On optimal, minimal brdf sampling for reflectance acquisition,” ACM Transactions on Graphics (TOG), vol. 34, no. 6, pp. 186:1–186:11, November 2015.
  13. D. Guarnera, G. C. Guarnera, A. Ghosh, C. Denk, and M. Glencross, “Brdf representation and acquisition,” in Proceedings of the 37th Annual Conference of the European Association for Computer Graphics: State of the Art Reports, ser. EG ’16.   Goslar Germany, Germany: Eurographics Association, 2016, pp. 625–650.
  14. S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and D. P. Greenberg, “Image-based brdf measurement including human skin,” in Proceedings of the 10th Eurographics Conference on Rendering, ser. EGWR’99.   Goslar, DEU: Eurographics Association, 1999, pp. 131––144.
  15. A. Gardner, C. Tchou, T. Hawkins, and P. Debevec, “Linear light source reflectometry,” ACM Trans. Graph., vol. 22, no. 3, pp. 749––758, jul 2003.
  16. B. Tunwattanapong, G. Fyffe, P. Graham, J. Busch, X. Yu, A. Ghosh, and P. Debevec, “Acquiring reflectance and shape from continuous spherical harmonic illumination,” ACM Trans. Graph., vol. 32, no. 4, jul 2013.
  17. T. Tongbuasirilai, J. Unger, J. Kronander, and M. Kurt, “Compact and intuitive data-driven brdf models,” The Visual Computer, vol. 36, pp. 855––872, May 2019.
  18. F. Romeiro, Y. Vasilyev, and T. Zickler, “Passive reflectometry,” in Computer Vision – ECCV 2008, D. Forsyth, P. Torr, and A. Zisserman, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 859–872.
  19. F. Romeiro and T. Zickler, “Blind reflectometry,” in Computer Vision – ECCV 2010, K. Daniilidis, P. Maragos, and N. Paragios, Eds.   Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 45–58.
  20. M. Aittala, T. Weyrich, and J. Lehtinen, “Two-shot svbrdf capture for stationary materials,” ACM Trans. Graph., vol. 34, no. 4, jul 2015.
  21. Z. Chen, S. Nobuhara, and K. Nishino, “Invertible neural brdf for object inverse rendering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 9380–9395, 2022.
  22. M. Boss, R. Braun, V. Jampani, J. T. Barron, C. Liu, and H. P. Lensch, “Nerd: Neural reflectance decomposition from image collections,” in IEEE/CVF International Conference on Computer Vision, 2021, pp. 12 664–12 674.
  23. M. Boss and H. P. A. Lensch, “Single image BRDF parameter estimation with a conditional adversarial network,” CoRR, vol. abs/1910.05148, 2019.
  24. D. Gao, X. Li, Y. Dong, P. Peers, K. Xu, and X. Tong, “Deep inverse rendering for high-resolution svbrdf estimation from an arbitrary number of images,” ACM Trans. Graph., vol. 38, no. 4, jul 2019.
  25. Y. Guo, C. Smith, M. Hašan, K. Sunkavalli, and S. Zhao, “Materialgan: Reflectance capture using a generative svbrdf model,” ACM Trans. Graph., vol. 39, no. 6, nov 2020.
  26. X. Li, Y. Dong, P. Peers, and X. Tong, “Modeling surface appearance from a single photograph using self-augmented convolutional neural networks,” ACM Transactions on Graphics (ToG), vol. 36, no. 4, pp. 1–11, 2017.
  27. Z. Li, K. Sunkavalli, and M. Chandraker, “Materials for masses: Svbrdf acquisition with a single mobile phone image,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 72–87.
  28. Z. Xu, K. Sunkavalli, S. Hadap, and R. Ramamoorthi, “Deep image-based relighting from optimal sparse samples,” ACM Transactions on Graphics (ToG), vol. 37, no. 4, pp. 1–13, 2018.
  29. K. Kang, Z. Chen, J. Wang, K. Zhou, and H. Wu, “Efficient reflectance capture using an autoencoder,” ACM Trans. Graph., vol. 37, no. 4, pp. 127–1, 2018.
  30. K. Kang, C. Xie, C. He, M. Yi, M. Gu, Z. Chen, K. Zhou, and H. Wu, “Learning efficient illumination multiplexing for joint capture of reflectance and shape,” ACM Trans. Graph., vol. 38, no. 6, 2019.
  31. Z. Xu, J. B. Nielsen, J. Yu, H. W. Jensen, and R. Ramamoorthi, “Minimal brdf sampling for two-shot near-field reflectance acquisition,” ACM Trans. Graph., vol. 35, no. 6, nov 2016.
  32. J. Yu, Z. Xu, M. Mannino, H. W. Jensen, and R. Ramamoorthi, “Sparse sampling for image-based svbrdf acquisition,” in Proceedings of the Eurographics 2016 Workshop on Material Appearance Modeling, ser. MAM ’16.   Goslar, DEU: Eurographics Association, 2016, p. 19–22.
  33. J. Lawrence, S. Rusinkiewicz, and R. Ramamoorthi, “Efficient brdf importance sampling using a factored representation,” in ACM SIGGRAPH 2004 Papers, ser. SIGGRAPH ’04.   New York, NY, USA: ACM, 2004, pp. 496––505.
  34. C. Soler, K. Subr, and D. Nowrouzezahrai, “A Versatile Parameterization for Measured Material Manifolds,” Computer Graphics Forum, vol. 37, no. 2, pp. 135–144, Apr. 2018.
  35. A. Bilgili, A. Öztürk, and M. Kurt, “A general brdf representation based on tensor decomposition,” Computer Graphics Forum, vol. 30, no. 8, pp. 2427–2439, 2011.
  36. J. Kautz and M. D. McCool, “Interactive rendering with arbitrary brdfs using separable approximations,” in Proceedings of the 10th Eurographics Conference on Rendering.   Eurographics Association, 1999, pp. 247–260.
  37. A. Ben-Artzi, R. Overbeck, and R. Ramamoorthi, “Real-time brdf editing in complex lighting,” ACM Trans. Graph., vol. 25, no. 3, pp. 945–954, jul 2006.
  38. T. Sun, H. W. Jensen, and R. Ramamoorthi, “Connecting measured brdfs to analytic brdfs by data-driven diffuse-specular separation,” ACM Trans. Graph., vol. 37, no. 6, dec 2018.
  39. M. M. Bagher, J. Snyder, and D. Nowrouzezahrai, “A non-parametric factor microfacet model for isotropic brdfs,” ACM Trans. Graph., vol. 35, no. 5, pp. 159:1–159:16, jul 2016.
  40. A. Sztrajman, G. Rainer, T. Ritschel, and T. Weyrich, “Neural brdf representation and importance sampling,” in Computer Graphics Forum, vol. 40, no. 6.   Wiley Online Library, 2021, pp. 332–346.
  41. T. Tongbuasirilai, J. Unger, C. Guillemot, and E. Miandji, “A sparse non-parametric brdf model,” ACM Trans. Graph., vol. 41, no. 5, oct 2022.
  42. P. Peers, D. K. Mahajan, B. Lamond, A. Ghosh, W. Matusik, R. Ramamoorthi, and P. Debevec, “Compressive light transport sensing,” ACM Transactions on Graphics (TOG), vol. 28, no. 1, pp. 1–18, 2009.
  43. P. Sen and S. Darabi, “Compressive dual photography,” in Computer Graphics Forum, vol. 28, no. 2.   Wiley Online Library, 2009, pp. 609–618.
  44. E. Miandji, J. Kronander, and J. Unger, “Compressive image reconstruction in reduced union of subspaces,” Computer Graphics Forum, vol. 34, no. 2, pp. 33–44, 2015.
  45. P. Sen and S. Darabi, “Compressive rendering: A rendering application of compressed sensing,” IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 4, pp. 487–499, 2011.
  46. K. Marwah, G. Wetzstein, Y. Bando, and R. Raskar, “Compressive Light Field Photography Using Overcomplete Dictionaries and Optimized Projections,” ACM Transactions on Graphics, vol. 32, no. 4, pp. 46:1–46:12, 2013.
  47. S. Hajisharif, E. Miandji, C. Guillemot, and J. Unger, “Single sensor compressive light field video camera,” Computer Graphics Forum, vol. 39, no. 2, pp. 463–474, 2020.
  48. R. Wang, Z. Yang, L. Liu, J. Deng, and F. Chen, “Decoupling noise and features via weighted ℓ1subscriptℓ1\ell_{1}roman_ℓ start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-analysis compressed sensing,” ACM Trans. Graph., vol. 33, no. 2, apr 2014.
  49. X. Lin, Y. Liu, J. Wu, and Q. Dai, “Spatial-spectral encoded compressive hyperspectral imaging,” ACM Trans. Graph., vol. 33, no. 6, nov 2014.
  50. N. Seylan, S. Ergun, and A. Öztürk, “Brdf reconstruction using compressive sensing,” in WSCG 2013.   Václav Skala-UNION Agency, 2013, pp. 88–94.
  51. B. Zupancic and C. Soler, “Sparse brdf approximation using compressive sensing,” in SIGGRAPH Asia 2013 Posters, ser. SA ’13.   New York, NY, USA: ACM, 2013.
  52. E. Miandji, “Sparse representation of visual data for compression and compressed sensing,” Ph.D. dissertation, Linköping University, 2018.
  53. E. Miandji, S. Hajisharif, and J. Unger, “A unified framework for compression and compressed sensing of light fields and light field videos,” ACM Trans. Graph., vol. 38, no. 3, may 2019.
  54. H. Otani, T. Komuro, S. Yamamoto, and N. Tsumura, “Bivariate brdf estimation based on compressed sensing,” in Advances in Computer Graphics, M. Gavrilova, J. Chang, N. M. Thalmann, E. Hitzer, and H. Ishikawa, Eds.   Springer, 2019, pp. 483–489.
  55. M. Elad, “Optimized projections for compressed sensing,” IEEE Transactions on Signal Processing, vol. 55, no. 12, pp. 5695–5702, 2007.
  56. Y. Xu, W. Yin, and S. Osher, “Learning circulant sensing kernels,” Inverse Problems and Imaging, vol. 8, no. 3, pp. 901–923, 2014.
  57. R. Rubinstein and M. Elad, “Analysis and synthesis sparse modeling methods in image processing,” Ph.D. dissertation, Computer Science Department, Technion, 2012.
  58. J. A. Tropp, A. C. Gilbert, and M. J. Strauss, “Algorithms for simultaneous sparse approximation. part i: Greedy pursuit,” Signal Processing, vol. 86, no. 3, pp. 572–588, 2006, sparse Approximations in Signal and Image Processing.
  59. J. Kim, J. Wang, L. T. Nguyen, and B. Shim, “Joint sparse recovery using signal space matching pursuit,” IEEE Transactions on Information Theory, vol. 66, no. 8, pp. 5072–5096, 2020.
  60. N. Han, S. Li, and J. Lu, “Orthogonal subspace based fast iterative thresholding algorithms for joint sparsity recovery,” IEEE Signal Processing Letters, vol. 28, no. 1, pp. 1320–1324, 2021.
  61. E. Miandji, M. Emadi, J. Unger, and E. Afshari, “On probability of support recovery for orthogonal matching pursuit using mutual coherence,” IEEE Signal Processing Letters, vol. 24, no. 11, pp. 1646–1650, nov 2017.
  62. J. Tropp, “Greed is good: algorithmic results for sparse approximation,” Information Theory, IEEE Transactions on, vol. 50, no. 10, pp. 2231–2242, oct 2004.
  63. M. Aharon, M. Elad, and A. M. Bruckstein, “K-svd: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Transactions on Signal Processing, vol. 54, no. 11, pp. 4311–4322, 2006.
  64. J. Löw, J. Kronander, A. Ynnerman, and J. Unger, “Brdf models for accurate and efficient rendering of glossy surfaces,” ACM Trans. Graph., vol. 31, no. 1, feb 2012.
  65. V. Blanz, A. Mehl, T. Vetter, and H.-P. Seidel, “A statistical method for robust 3d surface reconstruction from sparse data,” in Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT).   IEEE, 2004, pp. 293–300.
  66. W. Jakob, E. d’Eon, O. Jakob, and S. Marschner, “A comprehensive framework for rendering layered materials,” ACM Trans. Graph., vol. 33, no. 4, jul 2014.
  67. P. Henzler, V. Deschaintre, N. J. Mitra, and T. Ritschel, “Generative modelling of brdf textures from flash images,” ACM Trans. Graph., vol. 40, no. 6, dec 2021.
  68. E. J. Candes and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 21–30, 2008.
  69. G. Lavoué, N. Bonneel, J.-P. Farrugia, and C. Soler, “Perceptual quality of brdf approximations: dataset and metrics,” Computer Graphics Forum, vol. 40, no. 2, May 2021.
  70. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the reflectance field of a human face,” in Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’00.   USA: ACM Press/Addison-Wesley Publishing Co., 2000, p. 145–156.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now