Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Attentive Illumination Decomposition Model for Multi-Illuminant White Balancing (2402.18277v1)

Published 28 Feb 2024 in cs.CV

Abstract: White balance (WB) algorithms in many commercial cameras assume single and uniform illumination, leading to undesirable results when multiple lighting sources with different chromaticities exist in the scene. Prior research on multi-illuminant WB typically predicts illumination at the pixel level without fully grasping the scene's actual lighting conditions, including the number and color of light sources. This often results in unnatural outcomes lacking in overall consistency. To handle this problem, we present a deep white balancing model that leverages the slot attention, where each slot is in charge of representing individual illuminants. This design enables the model to generate chromaticities and weight maps for individual illuminants, which are then fused to compose the final illumination map. Furthermore, we propose the centroid-matching loss, which regulates the activation of each slot based on the color range, thereby enhancing the model to separate illumination more effectively. Our method achieves the state-of-the-art performance on both single- and multi-illuminant WB benchmarks, and also offers additional information such as the number of illuminants in the scene and their chromaticity. This capability allows for illumination editing, an application not feasible with prior methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. When color constancy goes wrong: Correcting improperly white-balanced images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1535–1544, 2019.
  2. Auto white-balance correction for mixed-illuminant scenes. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1210–1219, 2022.
  3. Jonathan T Barron. Convolutional color constancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 379–387, 2015.
  4. Fast fourier color constancy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 886–894, 2017.
  5. Multi-illuminant estimation with conditional random fields. IEEE Transactions on Image Processing (TIP), 23(1):83–96, 2013.
  6. Adaptive color constancy using faces. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36(8):1505–1518, 2014.
  7. Color constancy using cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop (CVPR - Workshop), pages 81–89, 2015.
  8. Single and multiple illuminant estimation using convolutional neural networks. IEEE Transactions on Image Processing (TIP), 26(9):4347–4362, 2017.
  9. Color constancy and non-uniform illumination: Can existing algorithms work? In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshop (ICCV - Workshop), pages 774–781. IEEE, 2011.
  10. Gershon Buchsbaum. A spatial processor model for object colour perception. Journal of the Franklin institute, 310(1):1–26, 1980.
  11. Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. Journal of the Optical Society of America A (JOSA A), 31(5):1049–1058, 2014.
  12. Empirical evaluation of gated recurrent neural networks on sequence modeling. Advances in Neural Information Processing Systems Workshop (NeurIPS - Workshop), 2014.
  13. A large image database for color constancy research. In Color and Imaging Conference, pages 160–164. Society for Imaging Science and Technology, 2003.
  14. Genesis: Generative scene inference and sampling with object-centric latent representations. arXiv preprint arXiv:1907.13052, 2019.
  15. Shades of gray and colour constancy. In Color and Imaging Conference, pages 37–41. Society for Imaging Science and Technology, 2004.
  16. David A Forsyth. A novel algorithm for color constancy. International Journal of Computer Vision (IJCV), 5(1):5–35, 1990.
  17. Bayesian color constancy revisited. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1–8. IEEE, 2008.
  18. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4):1–12, 2017.
  19. Generalized gamut mapping using image derivative structures for color constancy. International Journal of Computer Vision (IJCV), 86(2-3):127–139, 2010.
  20. Improving color constancy by photometric edge weighting. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 34(5):918–929, 2011a.
  21. Color constancy for multiple light sources. IEEE Transactions on Image Processing (TIP), 21(2):697–707, 2011b.
  22. A multi-illuminant synthetic image test set. Color Research & Application, 45(6):1055–1066, 2020.
  23. Light mixture estimation for spatially varying white balance. In ACM SIGGRAPH, pages 1–7, 2008.
  24. Fc4: Fully convolutional color constancy with confidence-weighted pooling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4085–4094, 2017.
  25. White balance under mixed illumination using flash photography. In International Conference on Computational Photography (ICCP), pages 1–10. IEEE, 2016.
  26. Illuminant spectra-based source separation using flash photography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6209–6218, 2018.
  27. Large scale multi-illuminant (lsmi) dataset for developing white balance algorithm under mixed illumination. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2410–2419, 2021.
  28. Shepherding slots to objects: Towards stable and robust object-centric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19198–19207, 2023.
  29. Conditional object-centric learning from video. arXiv preprint arXiv:2111.12594, 2021.
  30. Edwin H Land. The retinex theory of color vision. Scientific american, 237(6):108–129, 1977.
  31. Transcc: Transformer-based multiple illuminant color constancy using multitask learning. arXiv preprint arXiv:2211.08772, 2022.
  32. Clcc: Contrastive learning for color constancy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8053–8063, 2021.
  33. Object-centric learning with slot attention. Advances in Neural Information Processing Systems, 33:11525–11538, 2020.
  34. A dataset of multi-illumination images in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4080–4089, 2019.
  35. Multiple illuminant color estimation via statistical inference on factor graphs. IEEE Transactions on Image Processing (TIP), 25(11):5383–5396, 2016.
  36. Approaching the computational color constancy as a classification problem through deep learning. Pattern Recognition (PR), 61:405–416, 2017.
  37. Recurrent color constancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 5458–5466, 2017.
  38. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234–241. Springer, 2015.
  39. Object scene representation transformer. arXiv preprint arXiv:2206.06922, 2022.
  40. Lilong Shi. Re-processed version of the gehler color constancy dataset of 568 images. http://www. cs. sfu. ca/~ color/data/, 2000.
  41. Deep specialized network for illuminant estimation. In Proceedings of Proceedings of European Conference on Computer Vision (ECCV), pages 371–387. Springer, 2016.
  42. Oleksii Sidorov. Conditional gans for multi-illuminant color constancy: Revolution or yet another approach? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop (CVPR - Workshop), pages 0–0, 2019.
  43. Edge-based color constancy. IEEE Transactions on Image Processing (TIP), 16(9):2207–2214, 2007.
  44. Language-mediated, object-centric representation learning. arXiv preprint arXiv:2012.15814, 2020.
  45. End-to-end illuminant estimation based on deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3616–3625, 2020.
  46. Cascading convolutional color constancy. In AAAI Conference on Artificial Intelligence (AAAI), pages 12725–12732, 2020.
  47. Slot-vps: Object-centric representation learning for video panoptic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3093–3103, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now