Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction (2304.12372v3)

Published 24 Apr 2023 in cs.CV

Abstract: Light plays an important role in human well-being. However, most computer vision tasks treat pixels without considering their relationship to physical luminance. To address this shortcoming, we introduce the Laval Photometric Indoor HDR Dataset, the first large-scale photometrically calibrated dataset of high dynamic range 360{\deg} panoramas. Our key contribution is the calibration of an existing, uncalibrated HDR Dataset. We do so by accurately capturing RAW bracketed exposures simultaneously with a professional photometric measurement device (chroma meter) for multiple scenes across a variety of lighting conditions. Using the resulting measurements, we establish the calibration coefficients to be applied to the HDR images. The resulting dataset is a rich representation of indoor scenes which displays a wide range of illuminance and color, and varied types of light sources. We exploit the dataset to introduce three novel tasks, where: per-pixel luminance, per-pixel color and planar illuminance can be predicted from a single input image. Finally, we also capture another smaller photometric dataset with a commercial 360{\deg} camera, to experiment on generalization across cameras. We are optimistic that the release of our datasets and associated code will spark interest in physically accurate light estimation within the community. Dataset and code are available at https://lvsn.github.io/beyondthepixel/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Polyhaven HDRIs. https://polyhaven.com/hdris, 2023.
  2. What else can fool deep learning? addressing color constancy errors on deep neural network performance. In Int. Conf. Comput. Vis., 2019.
  3. Deep white-balance editing. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  4. Auto white-balance correction for mixed-illuminant scenes. In IEEE Winter conf. App. Comput. Vis., 2022.
  5. Inverse tone mapping. In Int. Conf. Comput. Graph. Int. Tech., 2006.
  6. A framework for inverse tone mapping. The Vis. Comput., 23(7):467–478, 2007.
  7. Jonathan T Barron. Convolutional color constancy. In Int. Conf. Comput. Vis., 2015.
  8. Shape, illumination, and reflectance from shading. IEEE Trans. Pattern Anal. Mach. Intell., 37(8):1670–1687, 2014.
  9. Fast fourier color constancy. In IEEE Conf. Comput. Vis. Pattern Recog., 2017.
  10. Quasi-unsupervised color constancy. In IEEE Conf. Comput. Vis. Pattern Recog., 2019.
  11. Jane Brox. Brilliant: the evolution of artificial light. Houghton Mifflin Harcourt, 2010.
  12. Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. JOSA A, 31(5):1049–1058, 2014.
  13. Visual ecology. In Visual Ecology. Princeton University Press, 2014.
  14. Commission Internationale de L’Eclairage (CIE). Ilv: International lighting vocabulary, 2nd edition. Technical report, 2020.
  15. Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH. 2008.
  16. Effect of light on human circadian physiology. Sleep medicine clinics, 4(2):165–177, 2009.
  17. Deep reverse tone mapping. ACM Trans. Graph., 36(6), 2017.
  18. Color correction using root-polynomial regression. IEEE Trans. Image Process., 24(5):1460–1470, 2015.
  19. Deep parametric indoor lighting estimation. In Int. Conf. Comput. Vis., 2019.
  20. Learning to predict indoor illumination from a single image. ACM Trans. Graph., 9(4), 2017.
  21. Bayesian color constancy revisited. In IEEE Conf. Comput. Vis. Pattern Recog., 2008.
  22. Casual indoor HDR radiance capture from omnidirectional images. In Brit. Mach. Vis. Conf., 2022.
  23. Color constancy for multiple light sources. IEEE Trans. Image Process., 21(2):697–707, 2011.
  24. Automatic noise modeling for ghost-free hdr reconstruction. ACM Trans. Graph., 32(6):1–10, 2013.
  25. Outcast: Single image relighting with cast shadows. Comput. Graph. Forum, 43, 2022.
  26. Modeling the space of camera response functions. IEEE Trans. Pattern Anal. Mach. Intell., 26(10):1272–1282, 2004.
  27. Recovering real-world reflectance properties and shading from HDR imagery. In Int. conf. 3D Vis., 2021.
  28. Deep residual learning for image recognition. In IEEE Conf. Comput. Vis. Pattern Recog., 2016.
  29. FlexISP: A flexible camera image processing framework. ACM Trans. Graph., 33(6), 2014.
  30. Light mixture estimation for spatially varying white balance. In ACM Trans. Graph., volume 27, pages 1–7. 2008.
  31. Hdr-nerf: High dynamic range neural radiance fields. In IEEE Conf. Comput. Vis. Pattern Recog., 2022.
  32. Measuring circadian lighting through high dynamic range photography. Lighting Res. & Tech., 51(5):742–763, 2019.
  33. Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph., 36(4), 2017.
  34. A software platform for manipulating the camera imaging pipeline. In Eur. Conf. Comput. Vis., 2016.
  35. Radiometric calibration with illumination change for outdoor scene analysis. In IEEE Conf. Comput. Vis. Pattern Recog., 2008.
  36. Robust radiometric calibration and vignetting correction. IEEE Trans. Pattern Anal. Mach. Intell., 30(4):562–576, 2008.
  37. Content-adaptive inverse tone mapping. In Vis. Comm. Image Proc., 2012.
  38. High dynamic range and super-resolution from raw image bursts. ACM Trans. Graph., 41(4), jul 2022.
  39. Deep chain HDRI: Reconstructing a high dynamic range image from a single low dynamic range image. IEEE Access, 2018.
  40. A multiple-camera system calibration toolbox using a feature descriptor-based calibration pattern. In IEEE/RSJ Int. Conf. Intell. Robots Syst., 2013.
  41. HDRNET: Single-image-based hdr reconstruction using channel attention cnn. In Int. Conf. Multimedia Sys. Sig. Process, 2019.
  42. Revisiting radiometric calibration for color computer vision. In Int. Conf. Comput. Vis., 2011.
  43. Radiometric calibration from a single image. In IEEE Conf. Comput. Vis. Pattern Recog., 2004.
  44. Determining the radiometric response function from a single grayscale image. In IEEE Conf. Comput. Vis. Pattern Recog., 2005.
  45. Bruce Lindbloom. XYZ to RGB. http://www.brucelindbloom.com/Eqn_XYZ_to_RGB.html, 2017.
  46. Single-image HDR reconstruction by learning to reverse the camera pipeline. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  47. A 360⁢°360°360 ⁢ ° omnidirectional photometer using a Ricoh Theta Z1. In Color Imag. Conf., 2022.
  48. Seasonal affective disorder: an overview. Chronobiology international, 20(2):189–207, 2003.
  49. Hdr-vdp-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph., 30(4), jul 2011.
  50. ExpandNet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content. Comput. Graph. Forum, 37(2), 2018.
  51. C. S. McCamy. Correlated color temperature as an explicit function of chromaticity coordinates. Color Research & Application, 17(2):142–144, 1992.
  52. Single view point omnidirectional camera calibration from planar grids. In Int. Conf. Robot. Aut., 2007.
  53. Deep optics for single-shot high-dynamic-range imaging. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  54. Nerf in the dark: High dynamic range view synthesis from noisy raw images. In IEEE Conf. Comput. Vis. Pattern Recog., 2022.
  55. Nerf: Representing scenes as neural radiance fields for view synthesis. Comm. of the ACM, 65(1):99–106, 2021.
  56. Visual perception and natural illumination. Current Opinion in Behavioral Sciences, 30:48–54, 2019.
  57. Learning srgb-to-raw-rgb de-rendering with content-aware metadata. In IEEE Conf. Comput. Vis. Pattern Recog., 2022.
  58. HDR-GAN: HDR image reconstruction from multi-exposed LDR images with large motions. IEEE Trans. Image Process., 30:3885–3896, 2021.
  59. Tutorial: Luminance maps for daylighting studies from high dynamic range photography. LEUKOS, 17(2):140–169, 2021.
  60. Charles Poynton. Digital Video and HD: Algorithms and Interfaces, page 275. Morgan Kaufmann, 2 edition, 2012.
  61. Photographic tone reproduction for digital images. ACM Trans. Graph., 21(3):267–276, 2002.
  62. LDR2HDR: On-the-fly reverse tone mapping of legacy video and photographs. ACM Trans. Graph., 26(3):39, 2007.
  63. Dynamic range improvement through multiple exposures. In IEEE Int. Conf. Image Process., 1999.
  64. U-net: Convolutional networks for biomedical image segmentation. In Med. Img. Comp. Comput.-Ass. Int., 2015.
  65. Single image hdr reconstruction using a cnn with masked features and perceptual loss. ACM Trans. Graph., 39(4), 2020.
  66. Deep specialized network for illuminant estimation. In Eur. Conf. Comput. Vis., 2016.
  67. Learning rank-1 diffractive optics for single-shot high dynamic range imaging. In IEEE Conf. Comput. Vis. Pattern Recog., 2020.
  68. Greg Ward. Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures. ACM Trans. Graph., 8(2):17–30, 2003.
  69. Beyond visual attractiveness: Physically plausible single image hdr reconstruction for spherical panoramas. ArXiv, abs/2103.12926, 2021.
  70. From pixels to physics: Probabilistic color de-rendering. In IEEE Conf. Comput. Vis. Pattern Recog., 2012.
  71. Luminance attentive networks for hdr image and panorama reconstruction. Comput. Graph. Forum, 40(7), 2021.
  72. Fixup initialization: Residual learning without normalization. In Int. Conf. Learn. Represent., 2019.
  73. Learning high dynamic range from outdoor panoramas. In Int. Conf. Comput. Vis., 2017.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction