2000 character limit reached
An Inter-observer consistent deep adversarial training for visual scanpath prediction (2211.07336v2)
Published 14 Nov 2022 in cs.CV
Abstract: The visual scanpath is a sequence of points through which the human gaze moves while exploring a scene. It represents the fundamental concepts upon which visual attention research is based. As a result, the ability to predict them has emerged as an important task in recent years. In this paper, we propose an inter-observer consistent adversarial training approach for scanpath prediction through a lightweight deep neural network. The adversarial method employs a discriminative neural network as a dynamic loss that is better suited to model the natural stochastic phenomenon while maintaining consistency between the distributions related to the subjective nature of scanpaths traversed by different observers. Through extensive testing, we show the competitiveness of our approach in regard to state-of-the-art methods.
- Neurobiology of attention, Elsevier, 2005.
- “Improving a vision indoor localization system by a saliency-guided detection,” in 2014 IEEE Visual Communications and Image Processing Conference, 2014, pp. 149–152.
- S. Jia and Y. Zhang, “Saliency-based deep convolutional neural network for no-reference image quality assessment,” Multimedia Tools and Applications, vol. 77, no. 12, pp. 14859–14872, 2018.
- El Hassouni M. et al. AbouelazizI., Chetouani A., “v,” in Neural Comput & Applic, 2020, vol. 32, p. 16589–16603.
- Aladine Chetouani, “Convolutional neural network and saliency selection for blind image quality assessment,” in 2018 25th IEEE International Conference on Image Processing (ICIP), 2018, pp. 2835–2839.
- “Blind robust 3d mesh watermarking based on mesh saliency and wavelet transform for copyright protection,” Information, vol. 10, no. 2, 2019.
- “Saliency driven perceptual image compression,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 227–236.
- “Satsal: A multi-level self-attention based architecture for visual saliency prediction,” IEEE Access, vol. 10, pp. 20701–20713, 2022.
- “Visual saliency guided complex image retrieval,” Pattern Recognition Letters, vol. 130, pp. 64–72, 2020.
- “Detecting colour vision deficiencies via webcam-based eye-tracking: A case study,” in Proceedings of the 2023 Symposium on Eye Tracking Research and Applications, New York, NY, USA, 2023, ETRA ’23, Association for Computing Machinery.
- “Satsal: A multi-level self-attention based architecture for visual saliency prediction,” IEEE Access, pp. 1–1, 2022.
- “Salypath360: Saliency and scanpath prediction framework for omnidirectional images,” Electronic Imaging, vol. 34, no. 11, pp. 168–1–168–1, 2022.
- “Computational modelling of visual attention,” Nature reviews neuroscience, vol. 2, no. 3, pp. 194–203, 2001.
- “Saccadic model of eye movements for free-viewing condition,” Vision research, vol. 116, pp. 152–164, 2015.
- “Pathgan: visual scanpath prediction with generative adversarial networks,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 0–0.
- “Gravitational laws of focus of attention,” IEEE transactions on pattern analysis and machine intelligence, 2019.
- “Human scanpath prediction based on deep convolutional saccadic model,” Neurocomputing, 2020.
- “Salypath: A deep-based architecture for visual attention prediction,” in 2021 IEEE International Conference on Image Processing (ICIP), 2021, pp. 1464–1468.
- “Self supervised scanpath prediction framework for painting images,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022, pp. 1538–1547.
- “A domain adaptive deep learning solution for scanpath prediction of paintings,” in Proceedings of the 19th International Conference on Content-Based Multimedia Indexing, New York, NY, USA, 2022, CBMI ’22, p. 57–63, Association for Computing Machinery.
- “The relationship between the locations of spatial features and those of fixations made during visual examination of briefly presented images.,” Spatial vision, 1996.
- “Salicon: Saliency in context.,” in CVPR. 2015, pp. 1072–1080, IEEE Computer Society.
- “Learning to predict where humans look,” in IEEE International Conference on Computer Vision (ICCV), 2009.
- “A benchmark of computational models of saliency to predict human fixations,” in MIT Technical Report, 2012.
- “It depends on how you look at it: Scanpath comparison in multiple dimensions with multimatch, a vector-based approach,” Behavior research methods, vol. 44, no. 4, pp. 1079–1100, 2012.