2000 character limit reached
Which Experimental Design is Better Suited for VQA Tasks? Eye Tracking Study on Cognitive Load, Performance, and Gaze Allocations (2404.04036v1)
Published 5 Apr 2024 in cs.HC
Abstract: We conducted an eye-tracking user study with 13 participants to investigate the influence of stimulus-question ordering and question modality on participants using visual question-answering (VQA) tasks. We examined cognitive load, task performance, and gaze allocations across five distinct experimental designs, aiming to identify setups that minimize the cognitive burden on participants. The collected performance and gaze data were analyzed using quantitative and qualitative methods. Our results indicate a significant impact of stimulus-question ordering on cognitive load and task performance, as well as a noteworthy effect of question modality on task performance. These findings offer insights for the experimental design of controlled user studies in visualization research.
- Investigating Cognitive Load in Energy Network Control Rooms: Recommendations for Future Designs. Frontiers in psychology 13 (2022), 812677. https://doi.org/10.3389/fpsyg.2022.812677
- VQA: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). IEEE, 2425–2433. https://doi.org/10.1109/iccv.2015.279
- Joseph L. Brooks. 2012. Counterbalancing for serial order carryover effects in experimental condition orders. Psychological Methods 17, 4 (2012), 600. https://doi.org/10.1037/a0029310
- Monica S Castelhano and John M Henderson. 2007. Initial scene representations facilitate eye movement guidance in visual search. Journal of Experimental Psychology: Human Perception and Performance 33, 4 (2007), 753–763. https://doi.org/10.1037/0096-1523.33.4.753
- Reading Strategies for Graph Visualizations that Wrap Around in Torus Topology. In Proceedings of the 2023 Symposium on Eye Tracking Research and Applications (Tubingen, Germany) (ETRA ’23). Association for Computing Machinery, New York, NY, USA, Article 67, 7 pages. https://doi.org/10.1145/3588015.3589841
- Gazealytics: A Unified and Flexible Visual Toolkit for Exploratory and Comparative Gaze Analysis. In Proceedings of the 2023 Symposium on Eye Tracking Research and Applications (Tubingen, Germany) (ETRA ’23). Association for Computing Machinery, New York, NY, USA, Article 69, 7 pages. https://doi.org/10.1145/3588015.3589844
- Eye activity as a measure of human mental effort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces (Palo Alto, CA, USA) (IUI ’11). Association for Computing Machinery, New York, NY, USA, 315–318. https://doi.org/10.1145/1943403.1943454
- Predicting human scanpaths in visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10876–10885. https://doi.org/10.1109/CVPR46437.2021.01073
- Structure of presented stimuli influences gazing behavior and choice. Food Quality and Preference 83 (2020), 103915. https://doi.org/10.1016/j.foodqual.2020.103915
- Sandra G. Hart. 2006. NASA-Task Load Index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50, 9 (2006), 904–908. https://doi.org/10.1177/154193120605000909
- Drew A. Hudson and Christopher D. Manning. 2019. GQA: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 6693–6702. https://doi.org/10.1109/CVPR.2019.00686
- AiR: Attention with reasoning capability. In Computer Vision – ECCV 2020. Springer International Publishing, Cham, 91–107. https://doi.org/10.1007/978-3-030-58452-8_6
- Gondy Leroy and David Kauchak. 2019. A comparison of text versus audio for information comprehension with future uses for smart speakers. JAMIA Open 2, 2 (2019), 254–260. https://doi.org/10.1093/jamiaopen/ooz011
- Measuring cognitive load in introductory CS: adaptation of an instrument. In Proceedings of the Tenth Annual Conference on International Computing Education Research (Glasgow, Scotland, United Kingdom) (ICER ’14). Association for Computing Machinery, New York, NY, USA, 131–138. https://doi.org/10.1145/2632320.2632348
- Comparative eye tracking study on node-link visualizations of trajectories. IEEE transactions on visualization and computer graphics 20, 12 (2014), 2221–2230. https://doi.org/10.1109/TVCG.2014.2346420
- Estimating cognitive load using remote eye tracking in a driving simulator. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications (Austin, Texas) (ETRA ’10). Association for Computing Machinery, New York, NY, USA, 141–144. https://doi.org/10.1145/1743666.1743701
- Helen C. Purchase. 2012. Experimental Human-Computer Interaction: A Practical Guide with Visual Examples. Vol. 9781107010062. Cambridge University Press, United Kingdom. https://doi.org/10.1017/CBO9780511844522
- Dario D. Salvucci and Joseph H. Goldberg. 2000. Identifying fixations and saccades in eye-tracking protocols. In Proceedings of the 2000 Symposium on Eye Tracking Research & Applications (Palm Beach Gardens, Florida, USA) (ETRA ’00). Association for Computing Machinery, New York, NY, USA, 71–78. https://doi.org/10.1145/355017.355028
- Text Comprehension: Heads-Up vs. Auditory Displays: Implications for a Productive Work Environment in SAE Level 3 Automated Vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New York, NY, USA, 342–354. https://doi.org/10.1145/3342197.3344547
- John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive Science 12, 2 (1988), 257–285. https://doi.org/10.1016/0364-0213(88)90023-7
- John Sweller. 2010a. Cognitive Load Theory: Recent Theoretical Advances. Cambridge University Press, 29–47. https://doi.org/10.1017/CBO9780511844744.004
- John Sweller. 2010b. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational psychology review 22 (2010), 123–138. https://doi.org/10.1007/s10648-010-9128-5
- Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. Quarterly Journal of Experimental Psychology Section A 57, 1 (2004), 165–182. https://doi.org/10.1080/02724980343000189
- Supplemental Material for “Which Experimental Design is Better Suited for VQA Tasks? – Eye Tracking Study on Cognitive Load, Performance, and Gaze Allocations”. https://doi.org/10.18419/darus-3380
- An eye-tracking study of website complexity from cognitive load perspective. Decision Support Systems 62 (2014), 1–10. https://doi.org/10.1016/j.dss.2014.02.007
- Impact of Gaze Uncertainty on AOIs in Information Visualisations. In 2022 Symposium on Eye Tracking Research and Applications (Seattle, WA, USA) (ETRA ’22). Association for Computing Machinery, New York, NY, USA, Article 60, 6 pages. https://doi.org/10.1145/3517031.3531166
- Jeremy M. Wolfe. 2021. Guided Search 6.0: An updated model of visual search. Psychonomic Bulletin & Review 28 (2021), 1060–1092. https://doi.org/10.3758/s13423-020-01859-9
Sponsored by Paperpile, the PDF & BibTeX manager trusted by top AI labs.
Get 30 days freePaper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.