Papers
Topics
Authors
Recent
2000 character limit reached

Towards a Perceptual Evaluation Framework for Lighting Estimation (2312.04334v3)

Published 7 Dec 2023 in cs.CV

Abstract: Progress in lighting estimation is tracked by computing existing image quality assessment (IQA) metrics on images from standard datasets. While this may appear to be a reasonable approach, we demonstrate that doing so does not correlate to human preference when the estimated lighting is used to relight a virtual scene into a real photograph. To study this, we design a controlled psychophysical experiment where human observers must choose their preference amongst rendered scenes lit using a set of lighting estimation algorithms selected from the recent literature, and use it to analyse how these algorithms perform according to human perception. Then, we demonstrate that none of the most popular IQA metrics from the literature, taken individually, correctly represent human perception. Finally, we show that by learning a combination of existing IQA metrics, we can more accurately represent human preference. This provides a new perceptual framework to help evaluate future lighting estimation algorithms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Image statistics do not explain the perception of gloss and lightness. J. Vis., 9(11):10, 2009.
  2. FLIP: a difference evaluator for alternating images. ACM Comp. Graph. Int. Tech., 3(2):15:1–15:23, 2020.
  3. Beyond the pixel: a photometrically calibrated HDR dataset for luminance and color prediction. In Int. Conf. Comput. Vis., 2023.
  4. Cues to an equivalent lighting model. J. Vis., 6(2):2–2, 2006.
  5. Brent Burley. Physically-based shading at disney. 2012.
  6. Learning scene illumination by pairwise photos from rear and front mobile cameras. Comput. Graph. Forum, 37(7):213–221, 2018.
  7. Everlight: Indoor-outdoor editable HDR lighting estimation. In Int. Conf. Comput. Vis., 2023.
  8. Image quality assessment: Unifying structure and texture similarity. IEEE Trans. Pattern Anal. Mach. Intell., 44(5):2567–2581, 2020.
  9. Locally adaptive structure and texture similarity for image quality assessment. In ACM Int. Conf. Multimedia, 2021.
  10. Image quality assessment using bsif, clbp, lcp, and lpq operators. Theoretical Computer Science, 805:37–61, 2020.
  11. Learning to predict indoor illumination from a single image. ACM Trans. Graph., 9(4), 2017.
  12. Deep parametric indoor lighting estimation. In Int. Conf. Comput. Vis., 2019.
  13. Fast spatially-varying indoor lighting estimation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019.
  14. Perceptual analysis of distance measures for color constancy algorithms. J. Opt. Soc. Am. A, 26(10):2243–2256, 2009.
  15. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Adv. Neural Inform. Process. Syst., page 6629–6640, 2017.
  16. Deep outdoor illumination estimation. In IEEE Conf. Comput. Vis. Pattern Recog., 2017.
  17. Image quality metrics: PSNR vs. SSIM. In Int. Conf. Pattern Recog., 2010.
  18. Image-based material editing. ACM Trans. Graph., 25(3):654–663, 2006.
  19. Light direction from shad(ow)ed random gaussian surfaces. Perception, 33(12):1405–1420, 2004.
  20. Lighting estimation in outdoor image collections. In Int. Conf. 3D Vis., 2014.
  21. Estimating the natural illumination conditions from a single outdoor image. Int. J. Comput. Vis., 98(2):123–145, 2012.
  22. Deeplight: Learning illumination for unconstrained mobile mixed reality. In IEEE Conf. Comput. Vis. Pattern Recog., 2019.
  23. Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and SVBRDF from a single image. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  24. Spatiotemporally consistent hdr indoor lighting estimation. ACM Trans. Graph., 42(3), 2023.
  25. Measuring the perception of light inconsistencies. In Symp. App. Percept. Graph. Vis., 2010.
  26. Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph., 30(4):1–14, 2011.
  27. The perception and misperception of specular surface reflectance. Cur. Biol., 22(20):1909–1913, 2012.
  28. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process., 21(12):4695–4708, 2012.
  29. Making a ”completely blind” image quality analyzer. IEEE Signal Process. Lett., 20(3):209–212, 2013.
  30. Ethan D Montag. Empirical formula for creating error bars for the method of paired comparison. J. Elec. Imag., 15(1):010502–010502, 2006.
  31. Visual perception and natural illumination. Curr. Opinion Behavioral Sc., 30:48–54, 2019.
  32. The influence of shape cues on the perception of lighting direction. J. Vis., 10(12):21–21, 2010.
  33. Chromatic illumination discrimination ability reveals that human colour constancy is optimised for blue daylight illuminations. PloS one, 9(2):e87989, 2014.
  34. John Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers, 10(3):61–74, 1999.
  35. Matching illumination of solid objects. Perception & psychophysics, 69(3):459–468, 2007.
  36. Pieapp: Perceptual image-error assessment through pairwise preference. In IEEE Conf. Comput. Vis. Pattern Recog., 2018.
  37. Visual equivalence: towards a new standard for image fidelity. ACM Trans. Graph., 26(3):76–es, 2007.
  38. High-resolution image synthesis with latent diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., 2022.
  39. Improved techniques for training gans. In Adv. Neural Inform. Process. Syst., page 2234–2242, 2016.
  40. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans. Image Process., 14(12):2117–2128, 2005.
  41. Blindly assess image quality in the wild guided by a self-adaptive hyper network. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  42. The perception of lighting inconsistencies in composite outdoor scenes. ACM Trans. Appl. Percept., 12(4):1–18, 2015.
  43. Perception of object illumination depends on highlights and shadows, not shading. J. Vis., 17(8):2–2, 2017.
  44. Louis L Thurstone. A law of comparative judgment. In Scaling, pages 81–92. Routledge, 2017.
  45. Color constancy algorithms: Psychophysical evaluation on a new dataset. J. Imaging Sci. Technol, 1(3):1, 2009.
  46. Stylelight: HDR panorama generation for lighting estimation and editing. In Eur. Conf. Comput. Vis., 2022.
  47. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4):600–612, 2004.
  48. Learning indoor inverse rendering with 3D spatially-varying lighting. In Int. Conf. Comput. Vis., 2021.
  49. Editable indoor lighting estimation. In Eur. Conf. Comput. Vis., 2022.
  50. Color science: concepts and methods, quantitative data and formulae. John Wiley & Sons, 2000.
  51. Vision models for wide color gamut imaging in cinema. IEEE Trans. Pattern Anal. Mach. Intell., 43(5):1777–1790, 2021.
  52. All-weather deep outdoor lighting estimation. In IEEE Conf. Comput. Vis. Pattern Recog., 2019.
  53. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE Conf. Comput. Vis. Pattern Recog., 2018.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Sign up for free to view the 1 tweet with 3 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free