Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Attention-Aware Visualization: Tracking and Responding to User Perception Over Time (2404.10732v2)

Published 16 Apr 2024 in cs.HC

Abstract: We propose the notion of Attention-Aware Visualizations (AAVs) that track the user's perception of a visual representation over time and feed this information back to the visualization. Such context awareness is particularly useful for ubiquitous and immersive analytics where knowing which embedded visualizations the user is looking at can be used to make visualizations react appropriately to the user's attention: for example, by highlighting data the user has not yet seen. We can separate the approach into three components: (1) measuring the user's gaze on a visualization and its parts; (2) tracking the user's attention over time; and (3) reactively modifying the visual representation based on the current attention metric. In this paper, we present two separate implementations of AAV: a 2D data-agnostic method for web-based visualizations that can use an embodied eyetracker to capture the user's gaze, and a 3D data-aware one that uses the stencil buffer to track the visibility of each individual mark in a visualization. Both methods provide similar mechanisms for accumulating attention over time and changing the appearance of marks in response. We also present results from a qualitative evaluation studying visual feedback and triggering mechanisms for capturing and revisualizing attention.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Context-awareness in wearable and ubiquitous computing. Virtual Reality, 3(3):200–211, sep 1998. doi: 10 . 1007/BF01408562
  2. S. S. Alam and R. Jianu. Analyzing eye-tracking information in visualization and data space: From where on the screen to what on the screen. IEEE Transactions on Visualization and Computer Graphics, 23(5):1492–1505, 2017. doi: 10 . 1109/TVCG . 2016 . 2535340
  3. Engineering context-aware systems and applications: A survey. Journal of Systems and Software, 117:55–83, 2016. doi: 10 . 1016/j . jss . 2016 . 02 . 010
  4. Context awareness in mobile computing environments. Wireless Personal Communications, 42(3):445–464, aug 2007. doi: 10 . 1007/s11277-006-9187-6
  5. Determining what individual sus scores mean: adding an adjective rating scale. Journal of Usability Studies, 4(3):114–123, may 2009.
  6. Triangulating user behavior using eye movement, interaction, and think aloud data. In Proceedings of the ACM Symposium on Eye Tracking Research & Applications, pp. 175–182. ACM, New York, NY, USA, 2016. doi: 10 . 1145/2857491 . 2857523
  7. State-of-the-art of visualization for eye tracking data. In State of the Art Reports of the Eurographics Conference on Visualization. Eurographics Association, 2014. doi: 10 . 2312/EUROVISSTAR . 20141173
  8. R. A. Bolt. Gaze-orchestrated dynamic windows. In Proceedings of the ACM Conference on Computer Graphics and Interactive Techniques, pp. 109–119. ACM, New York, NY, USA, 1981. doi: 10 . 1145/800224 . 806796
  9. D33{{}^{3}}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT Data-Driven Documents. 17(12):2301–2309, Dec. 2011. doi: 10 . 1109/TVCG . 2011 . 185
  10. J. Brooke et al. Sus-a quick and dirty usability scale. Usability evaluation in industry, 189(194):4–7, 1996.
  11. M. Burch. Eye Tracking and Visual Analytics. River Publishers, Gistrup, Denmark, 2021.
  12. Evaluation of traditional, orthogonal, and radial tree diagrams by an eye tracking study. IEEE Transactions on Visualization and Computer Graphics, 17(12):2440–2448, 2011. doi: 10 . 1109/TVCG . 2011 . 193
  13. A. K. Dey. Understanding and using context. Personal Ubiquitous Comput., 5(1):4–7, jan 2001. doi: 10 . 1007/s007790170019
  14. N. Elmqvist and P. Irani. Ubiquitous Analytics: Interacting with Big Data Anywhere, Anytime. Computer, 46(4):86–89, 2013. doi: 10 . 1109/MC . 2013 . 147
  15. N. Elmqvist and P. Tsigas. A taxonomy of 3D occlusion management for visualization. IEEE Transactions on Visualization and Computer Graphics, 14(5):1095–1109, 2008. doi: 10 . 1109/TVCG . 2008 . 59
  16. D. Fono and R. Vertegaal. EyeWindows: evaluation of eye-controlled zooming windows for focus selection. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 151–160. ACM, New York, NY, USA, 2005. doi: 10 . 1145/1054972 . 1054994
  17. VETA: visual eye-tracking analytics for the exploration of gaze patterns and behaviours. Visual Informatics, 6(2):1–13, 2022. doi: 10 . 1016/J . VISINF . 2022 . 02 . 004
  18. Towards pervasive augmented reality: Context-awareness in augmented reality. IEEE Transactions on Visualization and Computer Graphics, 23(6):1706–1724, 2017. doi: 10 . 1109/TVCG . 2016 . 2543720
  19. C. Gutwin and S. Greenberg. Design for individuals, design for groups: Tradeoffs between power and workspace awareness. In Proceedings of the ACM Conference on Computer-Supported Cooperative Work and Social Computing, pp. 207–216. ACM, New York, NY, USA, 1998. doi: 10 . 1145/289444 . 289495
  20. High-speed visual estimation using preattentive processing. ACM Transactions on Computer-Human Interaction, 6(2):107–135, 1999. doi: 10 . 1145/230562 . 230563
  21. Edit wear and read wear. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 3–9. ACM, New York, NY, USA, 1992. doi: 10 . 1145/142750 . 142751
  22. J. E. Hoffmann. A two-stage model of visual search. Perception & Psychophysics, 25:319–327, 1979. doi: 10 . 3758/BF03198811
  23. R. J. K. Jacob. What you look at is what you get: eye movement-based interaction techniques. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 11–18. ACM, New York, NY, USA, 1990. doi: 10 . 1145/97243 . 97246
  24. H. Jänicke and M. Chen. A salience-based quality metric for visualization. Computer Graphics Forum, 29(3):1183–1192, 2010. doi: 10 . 1111/J . 1467-8659 . 2009 . 01667 . X
  25. Does an eye tracker tell the truth about visualizations?: Findings while investigating visualizations for decision making. IEEE Transactions on Visualization and Computer Graphics, 18(12):2421–2430, 2012. doi: 10 . 1109/TVCG . 2012 . 215
  26. Eye tracking evaluation of visual analytics. Information Visualization, 15(4):340–358, 2016. doi: 10 . 1177/1473871615609787
  27. J. D. Mackinlay. Automating the design of graphical presentations of relational information. ACM Transactions on Graphics, 5(2):110–141, 1986. doi: 10 . 1145/22949 . 22950
  28. Data visualization saliency model: A tool for evaluating abstract data visualizations. IEEE Transactions on Visualization and Computer Graphics, 24(1):563–573, 2018. doi: 10 . 1109/TVCG . 2017 . 2743939
  29. T. Munzner. Visualization Analysis and Design. A K Peters, Boca Raton, FL, USA, 2014.
  30. U. Neisser. Cognitive Psychology. Appleton-Century-Crofts, New York, NY, USA, 1967.
  31. Change blindness in information visualization: A case study. In Proceedings of the IEEE Symposium on Information Visualization, pp. 15–22. IEEE Computer Society, Los Alamitos, CA, USA, 2001. doi: 10 . 1109/INFVIS . 2001 . 963274
  32. Context aware computing for the internet of things: A survey. IEEE Communications Surveys & Tutorials, 16(1):414–454, 2014. doi: 10 . 1109/SURV . 2013 . 042313 . 00197
  33. Gaze-touch: combining gaze with multi-touch for interaction on the same surface. In Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 509–518. ACM, New York, NY, USA, 2014. doi: 10 . 1145/2642918 . 2647397
  34. K. Pfeuffer and H. Gellersen. Gaze and touch interaction on tablets. In Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 301–311. ACM, New York, NY, USA, 2016. doi: 10 . 1145/2984511 . 2984514
  35. Gaze + pinch interaction in virtual reality. In Proceedings of the ACM Symposium on Spatial User Interaction, pp. 99–108. ACM, New York, NY, USA, 2017. doi: 10 . 1145/3131277 . 3132180
  36. Exploring visual attention and saliency modeling for task-based visual analysis. Computers & Graphics, 72:26–38, 2018. doi: 10 . 1016/j . cag . 2018 . 01 . 010
  37. Intelligent gaze-added interfaces. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 273–280. ACM, New York, NY, USA, 2000. doi: 10 . 1145/332040 . 332444
  38. Context-aware computing applications. In 1994 First Workshop on Mobile Computing Systems and Applications, pp. 85–90, 1994. doi: 10 . 1109/WMCSA . 1994 . 16
  39. The Reality of the Situation: A Survey of Situated Analytics. IEEE Transactions on Visualization and Computer Graphics, 2023. To appear. doi: 10 . 1109/TVCG . 2023 . 3285546
  40. A scanner deeply: Predicting gaze heatmaps on visualizations using crowdsourced eye movement data. IEEE Transactions on Visualization and Computer Graphics, 29(1):396–406, 2023. doi: 10 . 1109/TVCG . 2022 . 3209472
  41. S. Stellmach and R. Dachselt. Look & touch: gaze-supported target acquisition. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 2981–2990. ACM, New York, NY, USA, 2012. doi: 10 . 1145/2207676 . 2208709
  42. Designing gaze-supported multimodal interactions for the exploration of large image collections. In Proceedings of the Conference on Novel Gaze-Controlled Applications, pp. 1:1–1:8. ACM, New York, NY, USA, 2011. doi: 10 . 1145/1983302 . 1983303
  43. A. M. Treisman and G. Gelade. A feature-integration theory of attention. Cognitive Psychology, 12(1):97–136, 1980. doi: 10 . 1016/0010-0285(80)90005-5
  44. eSeeTrack - visualizing sequential fixation patterns. IEEE Transactions on Visualization and Computer Graphics, 16(6):953–962, 2010. doi: 10 . 1109/TVCG . 2010 . 149
  45. Eye pull, eye push: Moving objects between large screens and personal devices with gaze and touch. In Proceedings of the IFIP Conference on Human-Computer Interaction, vol. 8118 of Lecture Notes in Computer Science, pp. 170–186. Springer, New York, NY, USA, 2013. doi: 10 . 1007/978-3-642-40480-1_11
  46. An empirical investigation of gaze selection in mid-air gestural 3D manipulation. In Proceedings of the IFIP Conference on Human-Computer Interaction, vol. 9297 of Lecture Notes in Computer Science, pp. 315–330. Springer, New York, NY, USA, 2015. doi: 10 . 1007/978-3-319-22668-2_25
  47. C. Ware. Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco, CA, USA, 3 ed., 2012.
  48. C. Ware and H. H. Mikaelian. An evaluation of an eye tracker as a device for computer input. In Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, pp. 183–188. ACM, New York, NY, USA, 1987. doi: 10 . 1145/29933 . 275627
  49. M. Weiser. The computer for the 21st Century. Scientific American, 265(3):94–104, 1991. doi: 10 . 1145/329124 . 329126
  50. J. H. R. White and G. Buscher. User see, user point: Gaze and cursor alignment in web search. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 1341–1350. ACM, New York, NY, USA, 2012. doi: 10 . 1145/2207676 . 2208591
  51. J. M. Wolfe. Guided search 2.0 a revised model of visual search. Psychonomic Bulletin & Review, 1(2):202–238, 1994. doi: 10 . 3758/BF03200774
  52. Manual and gaze input cascaded (MAGIC) pointing. In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 246–253. ACM, New York, NY, USA, 1999. doi: 10 . 1145/302979 . 303053
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Arvind Srinivasan (8 papers)
  2. Johannes Ellemose (1 paper)
  3. Peter W. S. Butcher (3 papers)
  4. Panagiotis D. Ritsos (5 papers)
  5. Niklas Elmqvist (37 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets