2000 character limit reached
Self-Supervised Learning of Color Constancy (2404.08127v1)
Published 11 Apr 2024 in cs.CV, cs.AI, and cs.LG
Abstract: Color constancy (CC) describes the ability of the visual system to perceive an object as having a relatively constant color despite changes in lighting conditions. While CC and its limitations have been carefully characterized in humans, it is still unclear how the visual system acquires this ability during development. Here, we present a first study showing that CC develops in a neural network trained in a self-supervised manner through an invariance learning objective. During learning, objects are presented under changing illuminations, while the network aims to map subsequent views of the same object onto close-by latent representations. This gives rise to representations that are largely invariant to the illumination conditions, offering a plausible example of how CC could emerge during human cognitive development via a form of self-supervised learning.
- What else can fool deep learning? addressing color constancy errors on deep neural network performance. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), (2019).
- Time to augment self-supervised learning visual representation learning. In Eleventh International Conference on Learning Representations (ICLR), (2023).
- A cookbook of self-supervised learning, (2023).
- A simple framework for contrastive learning of visual representations. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR, (2020).
- Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019).
- (1987). A test of color constancy in 4-month-old human infants. Journal of Exp. Child Psychol., 44(2):255–267. 10.1016/0022-0965(87)90033-6.
- BlenderProc, (2019).
- A large-scale study on unsupervised spatiotemporal representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3299–3309, (2021).
- (2022). Deep neural models for color classification and color constancy. Journal of Vision, 22(4):17–17. 10.1167/jov.22.4.17.
- (2019). Learning to see stuff. Current Opinion in Behavioral Sciences, 30:100–108. 10.1016/j.cobeha.2019.07.004. Visual perception.
- Földiák, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3(2):194–200. 10.1162/neco.1991.3.2.194.
- Foster, D. H. (2011). Color constancy. Vision Research, 51(7):674–700. 10.1016/j.visres.2010.09.006.
- (2011). Invariant object recognition and pose estimation with slow feature analysis. Neural Computation, 23(9):2289–2323. 10.1162/NECO_a_00171.
- Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, (2019).
- Bootstrap your own latent - a new approach to self-supervised learning. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 21271–21284. Curran Associates, Inc., (2020).
- (2023). Object-based color constancy in a deep neural network. J. Opt. Soc. Am. A, 40(3):A48–A56. 10.1364/JOSAA.479451.
- (2017). Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph., 36(4). 10.1145/3072959.3073609.
- (1989). Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541–551. 10.1162/neco.1989.1.4.541.
- (2010). Unsupervised natural visual experience rapidly reshapes size-invariant object representation in inferior temporal cortex. Neuron, 67(6):1062–1075. 10.1016/j.neuron.2010.08.029.
- Clcc: Contrastive learning for color constancy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8053–8063, (2021).
- Deep learning from temporal coherence in video. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, page 737–744, New York, NY, USA, (2009). Association for Computing Machinery. ISBN 9781605585161. 10.1145/1553374.1553469.
- Mollon, J. D. (2003). The origins of modern color science. The Science of Color, 2:1–39.
- How useful is photo-realistic rendering for visual learning? In Hua, G. and Jégou, H., editors, Computer Vision – ECCV 2016 Workshops, pages 202–217, Cham, (2016). Springer International Publishing. ISBN 978-3-319-49409-8.
- The effectiveness of data augmentation in image classification using deep learning, (2017).
- (2020). Color constancy and color term knowledge are positively related during early childhood. Journal of Exp. Child Psychol., 196:104825. 10.1016/j.jecp.2020.104825.
- Contrastive learning through time. In SVRHM 2021 Workshop @ NeurIPS, (2021). URL https://openreview.net/forum?id=HTCRs8taN8.
- Steffy, G. Architectural lighting design. John Wiley & Sons, (2002).
- (2021). Learning about the world by learning about images. Current Directions in Psychological Science, 30(2):120–128. 10.1177/0963721421990334.
- (2021). Unsupervised learning predicts human perception and misperception of gloss. Nature Human Behaviour, 5(10):1402–1417. 10.1038/s41562-021-01097-6.
- Student. (1908). The probable error of a mean. Biometrika, 6(1):1–25. 10.2307/2331554.
- (2009). Learning illumination- and orientation-invariant representations of objects through temporal association. Journal of Vision, 9(7):6–6. 10.1167/9.7.6.
- (2021). Understanding how dimension reduction tools work: An empirical approach to deciphering t-SNE, UMAP, TriMap, and PaCMAP for data visualization. J. Mach. Learn. Res., 22(1).
- (2003). Is slowness a learning principle of the visual cortex? Zoology, 106(4):373–382. 10.1078/0944-2006-00132.
- (2018). The development of invariant object recognition requires visual experience with temporally smooth objects. Cognitive Science, 42(4):1391–1406. 10.1111/cogs.12595.