Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Bridging the Divide: Unraveling the Knowledge Gap in Data Visualization Research and Practice (2310.09614v2)

Published 14 Oct 2023 in cs.HC

Abstract: Empirical research on perception and cognition has laid the foundation for visualization design, often yielding useful design guidelines for practitioners. However, it remains uncertain how well practitioners stay informed about such crucial visualization design knowledge. In this paper, we employed a mixed-method approach to explore the knowledge gap between visualization research and real-world design guidelines. We initially collected existing design guidelines from various sources and empirical studies from diverse publishing venues, analyzing their alignment and uncovering missing links and inconsistent knowledge. Subsequently, we conducted surveys and interviews with practitioners and researchers to gain further insights into their experiences and attitudes towards design guidelines and empirical studies, and their views on the knowledge gap between research and practice. Our findings highlight the similarities and differences in their perspectives and propose strategies to bridge the divide in visualization design knowledge.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (74)
  1. L. Y.-H. Lo, A. Gupta, K. Shigyo, A. Wu, E. Bertini, and H. Qu, “Misinformed by visualization: What do we learn from misinformative visualizations?” Computer Graphics Forum, vol. 41, no. 3, pp. 515–525, 2022.
  2. W. S. Cleveland and R. McGill, “Graphical perception: Theory, experimentation, and application to the development of graphical methods,” Journal of the American statistical association, vol. 79, no. 387, pp. 531–554, 1984.
  3. S. Bateman, R. L. Mandryk, C. Gutwin, A. Genest, D. McDine, and C. Brooks, “Useful junk?: the effects of visual embellishment on comprehension and memorability of charts,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.   ACM, 2010, pp. 2573–2582.
  4. R. Borgo, A. Abdul-Rahman, F. Mohamed, P. W. Grant, I. Reppa, L. Floridi, and M. Chen, “An empirical study on using visual embellishments in visualization,” IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 12, pp. 2759–2768, 2012.
  5. M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh, D. Borkin, H. Pfister, and A. Oliva, “Beyond memorability: Visualization recognition and recall,” IEEE transactions on visualization and computer graphics, vol. 22, no. 1, pp. 519–528, 2015.
  6. E. Dimara, S. Franconeri, C. Plaisant, A. Bezerianos, and P. Dragicevic, “A task-based taxonomy of cognitive biases for information visualization,” IEEE transactions on visualization and computer graphics, 2018.
  7. J. Choi, C. Oh, Y.-S. Kim, and N. W. Kim, “Vislab: Enabling visualization designers to gather empirically informed design feedback,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, ser. CHI ’23.   New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3544548.3581132
  8. “Evidence-based practice,” https://en.wikipedia.org/wiki/Evidence-based_practice, accessed: 2022-12-20.
  9. P. Davies, “What is evidence-based education?” British journal of educational studies, vol. 47, no. 2, pp. 108–121, 1999.
  10. D. L. Sackett, W. M. Rosenberg, J. M. Gray, R. B. Haynes, and W. S. Richardson, “Evidence based medicine: what it is and what it isn’t,” pp. 71–72, 1996.
  11. H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale, “Empirical studies in information visualization: Seven scenarios,” IEEE transactions on visualization and computer graphics, vol. 18, no. 9, pp. 1520–1536, 2011.
  12. “Data visualization society,” https://www.datavisualizationsociety.com, Accessed: March 28, 2023.
  13. P. Parsons, “Understanding data visualization design practice,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 665–675, 2022.
  14. V. L. Ward, A. O. House, and S. Hamer, “Knowledge brokering: exploring the process of transferring knowledge into action,” BMC health services research, vol. 9, pp. 1–6, 2009.
  15. “Tableau - tableau official site,” https://www.tableau.com, Accessed: March 28, 2023.
  16. C. Stolte, D. Tang, and P. Hanrahan, “Polaris: A system for query, analysis, and visualization of multidimensional relational databases,” IEEE Transactions on Visualization and Computer Graphics, vol. 8, no. 1, pp. 52–65, 2002.
  17. M. Bostock, V. Ogievetsky, and J. Heer, “D³ data-driven documents,” IEEE Transactions on Visualization and Computer Graphics, vol. 17, no. 12, pp. 2301–2309, 2011.
  18. A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, “Vega-lite: A grammar of interactive graphics,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 1, pp. 341–350, 2017.
  19. B. Johnson and B. Shneiderman, “Tree-maps: A space filling approach to the visualization of hierarchical information structures,” Tech. Rep., 1998.
  20. A. Inselberg and B. Dimsdale, “Parallel coordinates: a tool for visualizing multi-dimensional geometry,” in Proceedings of the First IEEE Conference on Visualization: Visualization ‘90, 1990, pp. 361–378.
  21. S. C. S. Joyner, A. Riegelhuth, K. Garrity, Y.-S. Kim, and N. W. Kim, “Visualization accessibility in the wild: Challenges faced by visualization designers,” in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, ser. CHI ’22.   New York, NY, USA: Association for Computing Machinery, 2022. [Online]. Available: https://doi.org/10.1145/3491102.3517630
  22. A. Bigelow, S. Drucker, D. Fisher, and M. Meyer, “Reflections on how designers design with data,” in Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, ser. AVI ’14.   New York, NY, USA: Association for Computing Machinery, 2014, p. 17–24. [Online]. Available: https://doi.org/10.1145/2598153.2598175
  23. N. W. Kim, E. Schweickart, Z. Liu, M. Dontcheva, W. Li, J. Popovic, and H. Pfister, “Data-driven guides: Supporting expressive design for information graphics,” IEEE transactions on visualization and computer graphics, vol. 23, no. 1, pp. 491–500, 2016.
  24. Z. Liu, J. Thompson, A. Wilson, M. Dontcheva, J. Delorey, S. Grigg, B. Kerr, and J. Stasko, “Data illustrator: Augmenting vector design tools with lazy data binding for expressive visualization authoring,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems.   ACM, 2018, p. 123.
  25. K. A. Cook and J. J. Thomas, “Illuminating the path: The research and development agenda for visual analytics,” Pacific Northwest National Lab.(PNNL), Richland, WA (United States), Tech. Rep., 2005.
  26. M. Sedlmair, M. Meyer, and T. Munzner, “Design study methodology: Reflections from the trenches and the stacks,” IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 12, pp. 2431–2440, 2012.
  27. M. Brehmer and T. Munzner, “A multi-level typology of abstract visualization tasks,” IEEE transactions on visualization and computer graphics, vol. 19, no. 12, pp. 2376–2385, 2013.
  28. J. Heer and M. Bostock, “Crowdsourcing graphical perception: using mechanical turk to assess visualization design,” in Proceedings of the SIGCHI conference on human factors in computing systems.   ACM, 2010, pp. 203–212.
  29. Y. Kim and J. Heer, “Assessing effects of task and data distribution on the effectiveness of visual encodings,” in Computer Graphics Forum, vol. 37, no. 3.   Wiley Online Library, 2018, pp. 157–167.
  30. B. Saket, A. Endert, and Ç. Demiralp, “Task-based effectiveness of basic visualizations,” IEEE transactions on visualization and computer graphics, vol. 25, no. 7, pp. 2505–2512, 2018.
  31. “vega/voyager: Visualization tool for data exploration - github,” https://github.com/vega/voyager, Accessed: March 28, 2023.
  32. K. Hu, M. A. Bakker, S. Li, T. Kraska, and C. Hidalgo, “Vizml: A machine learning approach to visualization recommendation,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12.
  33. J. C. Roberts, C. Headleand, and P. D. Ritsos, “Sketching designs using the five design-sheet methodology,” IEEE Transactions on Visualization and Computer Graphics, vol. 22, no. 1, pp. 419–428, 2016.
  34. S. He and E. Adar, “Vizitcards: A card-based toolkit for infovis design education,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 1, pp. 561–570, 2017.
  35. S. Lee, S.-H. Kim, and B. C. Kwon, “Vlat: Development of a visualization literacy assessment test,” IEEE transactions on visualization and computer graphics, vol. 23, no. 1, pp. 551–560, 2016.
  36. K. Börner, A. Bueckle, and M. Ginda, “Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments,” Proceedings of the National Academy of Sciences, vol. 116, no. 6, pp. 1857–1864, 2019.
  37. B. Alper, N. H. Riche, F. Chevalier, J. Boy, and M. Sezgin, “Visualization literacy at elementary school,” in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017, pp. 5485–5497.
  38. J. Boy, R. A. Rensink, E. Bertini, and J.-D. Fekete, “A principled way of assessing visualization literacy,” IEEE transactions on visualization and computer graphics, vol. 20, no. 12, pp. 1963–1972, 2014.
  39. L. W. Ge, Y. Cui, and M. Kay, “Calvi: Critical thinking assessment for literacy in visualizations,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–18.
  40. A. Diehl, A. Abdul-Rahman, M. El-Assady, B. Bach, D. A. Keim, and M. Chen, “Visguides: A forum for discussing visualization guidelines,” in EuroVis (Short Papers), 2018, pp. 61–65.
  41. Z. Wang, L. Sundin, D. Murray-Rust, and B. Bach, “Cheat sheets for data visualization techniques,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–13.
  42. J. Scott-Brown, “Presenting visualization guideline collections,” 2nd IEEE VIS 2018 Workshop on the Creation, Curation, Critique and Conditioning of Principles and Guidelines in Visualization, 2018. [Online]. Available: https://c4pgv.dbvis.de/Scott-Brown_2018.pdf
  43. M. Meyer, M. Sedlmair, P. S. Quinan, and T. Munzner, “The nested blocks and guidelines model,” Information Visualization, vol. 14, no. 3, pp. 234–249, 2015. [Online]. Available: https://doi.org/10.1177/1473871613510429
  44. “Data to viz,” https://www.data-to-viz.com/, Accessed: February 25, 2023.
  45. “Data viz project,” https://datavizproject.com/, Accessed: February 25, 2023.
  46. “Adobe design system,” https://spectrum.adobe.com/, Accessed: March 22, 2023.
  47. “Google design system,” https://m2.material.io/design/, Accessed: March 22, 2023.
  48. “Multiple views: Visualization research explained,” https://medium.com/multiple-views-visualization-research-explained, accessed: 2023-03-21.
  49. “Data stories,” https://datastori.es/, accessed: 2023-03-21.
  50. Y. Zhang, Y. Sun, J. D. Gaggiano, N. Kumar, C. Andris, and A. G. Parker, “Visualization design practices in a crisis: Behind the scenes with covid-19 dashboard creators,” IEEE Transactions on Visualization and Computer Graphics, vol. 29, no. 1, pp. 1037–1047, 2023.
  51. W. R. Penuel, A.-R. Allen, C. E. Coburn, and C. Farrell, “Conceptualizing research–practice partnerships as joint work at boundaries,” Journal of Education for Students Placed at Risk (JESPAR), vol. 20, no. 1-2, pp. 182–197, 2015.
  52. “Datawrapper,” https://www.datawrapper.de/, Accessed: February 25, 2023.
  53. “Datajournalism.com,” https://datajournalism.com/, Accessed: February 25, 2023.
  54. M. Chen, G. Grinstein, C. R. Johnson, J. Kennedy, and M. Tory, “Pathways for theoretical advances in visualization,” IEEE Computer Graphics and Applications, vol. 37, no. 4, pp. 103–112, 2017.
  55. P. Isenberg, F. Heimerl, S. Koch, T. Isenberg, P. Xu, C. Stolper, M. Sedlmair, J. Chen, T. Möller, and J. Stasko, “vispubdata.org: A Metadata Collection about IEEE Visualization (VIS) Publications,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, 2017, to appear.
  56. “Visperception,” http://visperception.com/, accessed:2020-12-31.
  57. G. J. Quadri and P. Rosen, “A survey of perception-based visualization studies by task,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 12, pp. 5026–5048, 2022.
  58. W. J. Potter and D. Levine‐Donnerstein, “Rethinking validity and reliability in content analysis,” Journal of Applied Communication Research, vol. 27, no. 3, pp. 258–284, 1999.
  59. “Flowing data,” https://flowingdata.com/chart-types/, Accessed: February 25, 2023.
  60. R. Kosara, “An empire built on sand: Reexamining what we think we know about visualization,” in Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, ser. BELIV ’16.   New York, NY, USA: Association for Computing Machinery, 2016, p. 162–168. [Online]. Available: https://doi.org/10.1145/2993901.2993909
  61. Z. Zeng and L. Battle, “A review and collation of graphical perception knowledge for visualization recommendation,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–16.
  62. R. Amar, J. Eagan, and J. Stasko, “Low-level components of analytic activity in information visualization,” in IEEE Symposium on Information Visualization, 2005. INFOVIS 2005., 2005, pp. 111–117.
  63. M. Kraus, K. Angerbauer, J. Buchmüller, D. Schweitzer, D. A. Keim, M. Sedlmair, and J. Fuchs, “Assessing 2d and 3d heatmaps for comparative analysis: An empirical study,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, ser. CHI ’20.   New York, NY, USA: Association for Computing Machinery, 2020, p. 1–14. [Online]. Available: https://doi.org/10.1145/3313831.3376675
  64. S. Haroz, R. Kosara, and S. L. Franconeri, “Isotype visualization: Working memory, performance, and engagement with pictographs,” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ser. CHI ’15.   New York, NY, USA: Association for Computing Machinery, 2015, p. 1191–1200.
  65. Y. Liu and J. Heer, “Somewhere over the rainbow: An empirical assessment of quantitative colormaps,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ser. CHI ’18.   New York, NY, USA: Association for Computing Machinery, 2018, p. 1–12.
  66. “Storytelling with data - community forum,” https://community.storytellingwithdata.com/, Accessed: March 28, 2023.
  67. “Wtf visualizations,” https://viz.wtf/, Accessed: February 25, 2023.
  68. “Financial times visual vocabulary,” https://ft-interactive.github.io/visual-vocabulary/, Accessed: February 25, 2023.
  69. “The data visualisation catalogue,” https://datavizcatalogue.com/, Accessed: February 25, 2023.
  70. D. Moritz, C. Wang, G. L. Nelson, H. Lin, A. M. Smith, B. Howe, and J. Heer, “Formalizing visualization design knowledge as constraints: Actionable and extensible models in draco,” IEEE transactions on visualization and computer graphics, vol. 25, no. 1, pp. 438–448, 2018.
  71. Z. Zeng, J. Yang, D. Moritz, J. Heer, and L. Battle, “Too many cooks: Exploring how graphical perception studies influence visualization recommendations in draco,” IEEE Transactions on Visualization and Computer Graphics, 2023.
  72. A. Ko, “Sustainable peer review via incentive aligned markets,” 2023, [Online; posted 27-March-2023]. [Online]. Available: https://medium.com/bits-and-behavior/sustainable-peer-review-via-incentive-aligned-markets-a64ff726da56
  73. N. W. Kim, G. Myers, and B. Bach, “How good is chatgpt in giving advice on your visualization design?” arXiv preprint arXiv:2310.09617, 2023.
  74. C. E. Coburn and W. R. Penuel, “Research–practice partnerships in education: Outcomes, dynamics, and open questions,” Educational researcher, vol. 45, no. 1, pp. 48–54, 2016.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Nam Wook Kim (14 papers)
  2. Grace Myers (3 papers)
  3. Jinhan Choi (2 papers)
  4. Yoonsuh Cho (1 paper)
  5. Changhoon Oh (4 papers)
  6. Yea-Seul Kim (9 papers)