Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Decoupling Judgment and Decision Making: A Tale of Two Tails (2312.12921v2)

Published 20 Dec 2023 in cs.HC

Abstract: Is it true that if citizens understand hurricane probabilities, they will make more rational decisions for evacuation? Finding answers to such questions is not straightforward in the literature because the terms judgment and decision making are often used interchangeably. This terminology conflation leads to a lack of clarity on whether people make suboptimal decisions because of inaccurate judgments of information conveyed in visualizations or because they use alternative yet currently unknown heuristics. To decouple judgment from decision making, we review relevant concepts from the literature and present two preregistered experiments (N=601) to investigate if the task (judgment vs. decision making), the scenario (sports vs. humanitarian), and the visualization (quantile dotplots, density plots, probability bars) affect accuracy. While experiment 1 was inconclusive, we found evidence for a difference in experiment 2. Contrary to our expectations and previous research, which found decisions less accurate than their direct-equivalent judgments, our results pointed in the opposite direction. Our findings further revealed that decisions were less vulnerable to status-quo bias, suggesting decision makers may disfavor responses associated with inaction. We also found that both scenario and visualization types can influence peoples judgments and decisions. Although effect sizes are not large and results should be interpreted carefully, we conclude that judgments cannot be safely used as proxy tasks for decision making, and discuss implications for visualization research and beyond.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (87)
  1. L. M. Padilla, S. H. Creem-Regehr, M. Hegarty, and J. K. Stefanucci, “Decision making with visualizations: a cognitive framework across disciplines,” CRPI, vol. 3, pp. 29:1–29:25, 2018.
  2. L. Cheong, S. Bleisch, A. Kealy, K. Tolhurst, T. Wilkening, and M. Duckham, “Evaluating the impact of visualization of wildfire hazard upon decision-making under uncertainty,” IJGIS, vol. 30, no. 7, pp. 1377–1404, 2016.
  3. S. C. Castro, P. S. Quinan, H. Hosseinpour, and L. Padilla, “Examining effort in 1d uncertainty communication using individual differences in working memory and nasa-tlx,” IEEE TVCG, vol. 28, no. 1, pp. 411–421, 2022.
  4. N. Jardine, B. D. Ondov, N. Elmqvist, and S. Franconeri, “The perceptual proxies of visual comparison,” IEEE TVCG, vol. 26, no. 1, pp. 1012–1021, 2020.
  5. A. Kale, F. Nguyen, M. Kay, and J. Hullman, “Hypothetical outcome plots help untrained observers judge trends in ambiguous data,” IEEE TVCG, vol. 25, no. 1, pp. 892–902, 2019.
  6. A. Kale, M. Kay, and J. Hullman, “Visual reasoning strategies for effect size judgments and decisions,” IEEE TVCG, vol. 27, no. 2, pp. 272–282, 2021.
  7. CRC Press, 2014.
  8. M. A. Elliott, C. Nothelfer, C. Xiong, and D. A. Szafir, “A design space of vision science methods for visualization research,” IEEE TVCG, vol. 27, no. 2, pp. 1117–1127, 2021.
  9. C. C. Gramazio, D. H. Laidlaw, and K. B. Schloss, “Colorgorical: Creating discriminable and preferable color palettes for information visualization,” IEEE TVCG, vol. 23, no. 1, pp. 521–530, 2017.
  10. J. Talbot, J. Gerth, and P. Hanrahan, “An empirical model of slope ratio comparisons,” IEEE TVCG, vol. 18, no. 12, pp. 2613–2620, 2012.
  11. S. Mittelstädt, A. Stoffel, and D. A. Keim, “Methods for compensating contrast effects in information visualization,” Comput. Graphics Forum, vol. 33, no. 3, pp. 231–240, 2014.
  12. R. Veras and C. Collins, “Saliency deficit and motion outlier detection in animated scatterplots,” in Proc. CHI, pp. 541:1–541:12, ACM, 2019.
  13. S. Haroz and D. Whitney, “How capacity limits of attention influence information visualization effectiveness,” IEEE TVCG, vol. 18, no. 12, pp. 2402–2410, 2012.
  14. C. Xiong, C. R. Ceja, C. J. Ludwig, and S. Franconeri, “Biased average position estimates in line and bar graphs: Underestimation, overestimation, and perceptual pull,” IEEE TVCG, vol. 26, no. 1, pp. 301–310, 2020.
  15. A. Patil, G. Richer, C. Jermaine, D. Moritz, and J.-D. Fekete, “Studying early decision making with progressive bar charts,” IEEE TVCG, vol. 29, no. 1, pp. 407–417, 2023.
  16. L. Harrison, F. Yang, S. Franconeri, and R. Chang, “Ranking visualizations of correlation using weber’s law,” IEEE TVCG, vol. 20, no. 12, pp. 1943–1952, 2014.
  17. A. C. Telea, Data visualization: Principles and practice. CRC Press, 2014.
  18. P. Kayongo, G. Sun, J. Hartline, and J. Hullman, “Visualization equilibrium,” IEEE TVCG, vol. 28, no. 1, pp. 465–474, 2022.
  19. Y. Zhang, R. K. Bellamy, and W. A. Kellogg, “Designing information for remediating cognitive biases in decision-making,” in Proc. CHI, pp. 2211–2220, ACM, 2015.
  20. C. Xiong, J. Shapiro, J. Hullman, and S. Franconeri, “Illusion of causality in visualized data,” IEEE TVCG, vol. 26, no. 1, pp. 853–862, 2020.
  21. H. Song and D. A. Szafir, “Where’s my data? evaluating visualizations with missing data,” IEEE TVCG, vol. 25, no. 1, pp. 914–924, 2019.
  22. J. Schwarz and M. Morris, “Augmenting web pages and search results to support credibility assessment,” in Proc. CHI, pp. 1245–1254, ACM, 2011.
  23. J. Zhao, M. Glueck, P. Isenberg, F. Chevalier, and A. Khan, “Supporting handoff in asynchronous collaborative sensemaking using knowledge-transfer graphs,” IEEE TVCG, vol. 24, no. 1, pp. 340–350, 2018.
  24. M. Borkin, K. Gajos, A. Peters, D. Mitsouras, S. Melchionna, F. Rybicki, C. Feldman, and H. Pfister, “Evaluation of artery visualizations for heart disease diagnosis,” IEEE TVCG, vol. 17, no. 12, pp. 2479–2488, 2011.
  25. F. Ferstl, M. Kanzler, M. Rautenhaus, and R. Westermann, “Time-hierarchical clustering and visualization of weather forecast ensembles,” IEEE TVCG, vol. 23, no. 1, pp. 831–840, 2017.
  26. D. Holten, “Hierarchical edge bundles: Visualization of adjacency relations in hierarchical data,” IEEE TVCG, vol. 12, no. 5, pp. 741–748, 2006.
  27. N. Willems, H. Van De Wetering, and J. J. Van Wijk, “Visualization of vessel movements,” Comput. Graphics Forum, vol. 28, no. 3, pp. 959–966, 2009.
  28. P. E. Rauber, A. X. Falcao, and A. C. Telea, “Projections as visual aids for classification system design,” Information Visualization, vol. 17, no. 4, pp. 282–305, 2018.
  29. Esri Press, 1983.
  30. A.K. Peters, 2015.
  31. E. Dimara, S. Franconeri, C. Plaisant, A. Bezerianos, and P. Dragicevic, “A task-based taxonomy of cognitive biases for information visualization,” IEEE TVCG, vol. 26, no. 2, pp. 1413–1432, 2020.
  32. E. Dimara, A. Bezerianos, and P. Dragicevic, “Conceptual and methodological issues in evaluating multidimensional visualizations for decision support,” IEEE TVCG, vol. 24, no. 1, pp. 749–759, 2018.
  33. M. Fernandes, L. Walls, S. Munson, J. Hullman, and M. Kay, “Uncertainty displays using quantile dotplots or cdfs improve transit decision-making,” in Proc. CHI, pp. 144:1–144:12, ACM, 2018.
  34. E. Hindalong, J. Johnson, G. Carenini, and T. Munzner, “Abstractions for visualizing preferences in group decisions,” ACM HCI, vol. 6, no. CSCW1, pp. 49:1–49:44, 2022.
  35. A. Savikhin, R. Maciejewski, and D. S. Ebert, “Applied visual analytics for economic decision-making,” in Proc. VAST, pp. 107–114, IEEE, 2008.
  36. A. Narechania, A. Coscia, E. Wall, and A. Endert, “Lumos: Increasing awareness of analytic behavior during visual data analysis,” IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 1009–1018, 2022.
  37. L. Cibulski, J. Schmidt, and W. Aigner, “Reflections on visualization research projects in the manufacturing industry,” IEEE CG&A, vol. 42, no. 2, pp. 21–32, 2022.
  38. E. Dimara and J. Stasko, “A critical reflection on visualization research: Where do decision making tasks hide?,” IEEE TVCG, vol. 28, no. 1, pp. 1128–1138, 2022.
  39. E. Dimara, H. Zhang, M. Tory, and S. Franconeri, “The unmet data visualization needs of decision makers within organizations,” IEEE TVCG, vol. 28, no. 12, pp. 4101–4112, 2022.
  40. H.-F. Cheng, R. Wang, Z. Zhang, F. O’Connell, T. Gray, F. M. Harper, and H. Zhu, “Explaining decision-making algorithms through ui: Strategies to help non-expert stakeholders,” in Proc. CHI, pp. 559:1–559:12, ACM, 2019.
  41. F. Sperrle, M. El-Assady, G. Guo, R. Borgo, D. H. Chau, A. Endert, and D. Keim, “A survey of human-centered evaluations in human-centered machine learning,” Comput. Graphics Forum, vol. 40, no. 3, pp. 543–568, 2021.
  42. A. Chatzimparmpas, R. M. Martins, I. Jusufi, K. Kucher, F. Rossi, and A. Kerren, “The state of the art in enhancing trust in machine learning models with the use of visualizations,” Comput. Graphics Forum, vol. 39, no. 3, pp. 713–756, 2020.
  43. P. Pirolli and S. Card, “The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis,” in Proceedings of International Conference on Intelligence Analysis, vol. 5, pp. 2–4, 2005.
  44. K. Sedig, P. Parsons, and A. Babanski, “Towards a characterization of interactivity in visual analytics.,” J. Multim. Process. Technol., vol. 3, no. 1, pp. 12–28, 2012.
  45. IEEE Visualization, “Analytics and decisions area model,” 2022. http://ieeevis.org/year/2022/info/call-participation/area-model.
  46. E. Dimara, A. Bezerianos, and P. Dragicevic, “Narratives in crowdsourced evaluation of visualizations: A double-edged sword?,” in Proc. CHI, pp. 5475–5484, ACM, 2017.
  47. A. P. Association., “APA Dictionary of Psychology (2nd ed.),” 2015.
  48. Springer Science & Business Media, 2013.
  49. K. Eberhard, “The effects of visualization on judgment and decision-making: a systematic literature review,” Management Review Quarterly, vol. 73, no. 1, pp. 167–214, 2023.
  50. John Wiley & Sons, Inc., 2012.
  51. B. Fischhoff and S. B. Broomell, “Judgment and decision making,” Annual Review of Psychology, vol. 71, no. 1, pp. 331–355, 2020.
  52. E. Oral, R. Chawla, M. Wijkstra, N. Mahyar, and E. Dimara, “From information to choice: A critical inquiry into visualization tools for decision making,” IEEE TVCG, 2023. Preprint.
  53. H. A. Simon, The new science of management decision. Harper & Brothers, 1960.
  54. N. J. Wispinski, J. P. Gallivan, and C. S. Chapman, “Models, movements, and minds: bridging the gap between decision making and action,” Ann. N. Y. Acad. Sci., vol. 1464, no. 1, pp. 30–51, 2020.
  55. N. Gold, B. D. Pulford, and A. M. Colman, “Do as i say, don’t do as i do: Differences in moral judgments do not translate into differences in decisions in real-life trolley problems,” Journal of Economic Psychology, vol. 47, pp. 50–61, 2015.
  56. G. Karlsson, “A phenomenological psychological study of decision and choice,” Acta Psychologica, vol. 68, no. 1, pp. 7–25, 1988.
  57. C. Pletti, L. Lotto, G. Buodo, and M. Sarlo, “It’s immoral, but i’d do it! psychopathy traits affect decision-making in sacrificial dilemmas and in everyday moral situations,” Br. J. Psychol., vol. 108, no. 2, pp. 351–368, 2017.
  58. J. A. Hampton, “Concepts as prototypes,” vol. 46 of Psychology of Learning and Motivation, pp. 79–113, Academic Press, 2006.
  59. L. Decock and I. Douven, “What is graded membership?,” Noûs, vol. 48, no. 4, pp. 653–682, 2014.
  60. John Wiley & Sons, Ltd, 2015.
  61. E. R. Van Norman, P. M. Nelson, J.-E. Shin, and T. J. Christ, “An evaluation of the effects of graphic aids in improving decision accuracy in a continuous treatment design,” J. Behav. Educ., vol. 22, pp. 283–301, 2013.
  62. G. A. Gescheider, Psychophysics: the fundamentals. Psychology Press, 2013.
  63. D. B. Liston and L. S. Stone, “Saccadic brightness decisions do not use a difference model,” JOV, vol. 13, no. 8, pp. 1:1–1:10, 2013.
  64. S. K. Lynn, J. B. Wormwood, L. F. Barrett, and K. S. Quigley, “Decision making from economic and signal detection perspectives: development of an integrated framework,” Frontiers in psychology, vol. 6, pp. 952:1–952:14, 2015.
  65. V. H. Visschers, R. M. Meertens, W. W. Passchier, and N. N. De Vries, “Probability information in risk communication: a review of the research literature,” Risk Anal., vol. 29, no. 2, pp. 267–287, 2009.
  66. H. F. Chua, J. F. Yates, and P. Shah, “Risk avoidance: Graphs versus numbers,” M&C, vol. 34, pp. 399–410, 2006.
  67. E. R. Stone, J. F. Yates, and A. M. Parker, “Effects of numerical and graphical displays on professed risk-taking behavior.,” JEP:A, vol. 3, no. 4, pp. 243–256, 1997.
  68. E. R. Stone, W. R. Sieck, B. E. Bull, J. F. Yates, S. C. Parks, and C. J. Rush, “Foreground: background salience: Explaining the effects of graphical displays on risk avoidance,” Organizational Behavior and Human Decision Processes, vol. 90, no. 1, pp. 19–36, 2003.
  69. M. Galesic, R. Garcia-Retamero, and G. Gigerenzer, “Using icon arrays to communicate medical risks: Overcoming low numeracy.,” Health Psychol., vol. 28, no. 2, pp. 210–216, 2009.
  70. C. M. Wu, B. Meder, F. Filimon, and J. D. Nelson, “Asking better questions: How presentation formats influence information search.,” JEP:LMC, vol. 43, no. 8, pp. 1274–1297, 2017.
  71. L. Morais, Y. Jansen, N. Andrade, and P. Dragicevic, “Can anthropographics promote prosociality? a review and large-sample study,” in Proc. CHI, pp. 611:1–611:18, ACM, 2021.
  72. H. Zhang and L. T. Maloney, “Ubiquitous log odds: A common representation of probability and frequency distortion in perception, action, and cognition,” Frontiers in Neuroscience, vol. 6, pp. 1:1–1:14, 2012.
  73. E. T. Cokely, M. Galesic, E. Schulz, S. Ghazal, and R. Garcia-Retamero, “Measuring risk literacy: The berlin numeracy test.,” Judgment and Decision Making, vol. 7, pp. 25–47, 2012.
  74. G. Cumming, “The new statistics: Why and how,” Psychol. Sci., vol. 25, no. 1, pp. 7–29, 2014.
  75. 2016.
  76. L. Besançon and P. Dragicevic, “The continued prevalence of dichotomous inferences at CHI,” in Proc. CHI EA, pp. alt14:1–alt14:11, ACM, 2019.
  77. G. Cumming and S. Finch, “Inference by eye: Confidence intervals and how to read pictures of data.,” Am. Psychol., vol. 60, no. 2, pp. 170–180, 2005.
  78. M. Krzywinski and N. Altman, “Error bars: The meaning of error bars is often misinterpreted, as is the statistical significance of their overlap,” Nature Methods, vol. 10, pp. 921–923, 2013.
  79. K. N. Kirby and D. Gerlanc, “BootES: An R package for bootstrap confidence intervals on effect sizes,” Behav. Res., vol. 45, pp. 905–927, 2013.
  80. A. Gelman and H. Stern, “The difference between “significant” and “not significant” is not itself statistically significant,” The American Statistician, vol. 60, no. 4, pp. 328–331, 2006.
  81. S. Nieuwenhuis, B. U. Forstmann, and E.-J. Wagenmakers, “Erroneous analyses of interactions in neuroscience: A problem of significance,” Nat. Neurosci., vol. 14, pp. 1105–1107, 2011.
  82. B. Dy, N. Ibrahim, A. Poorthuis, and S. Joyce, “Improving visualization design for effective multi-objective decision making,” IEEE TVCG, vol. 28, no. 10, pp. 3405–3416, 2022.
  83. G. Lufityanto, C. Donkin, and J. Pearson, “Measuring intuition: Nonconscious emotional information boosts decision accuracy and confidence,” Psychol. Sci., vol. 27, no. 5, pp. 622–634, 2016.
  84. S. M. Fleming, C. L. Thomas, and R. J. Dolan, “Overcoming status quo bias in the human brain,” Proc. Natl. Acad. Sci., vol. 107, no. 13, pp. 6005–6009, 2010.
  85. D. Kahneman, “Article commentary: Judgment and decision making: A personal view,” APS, vol. 2, no. 3, pp. 142–145, 1991.
  86. D. Kahneman, “A perspective on judgment and choice: mapping bounded rationality.,” Am. Psychol., vol. 58, no. 9, pp. 697–720, 2003.
  87. G. Gigerenzer and W. Gaissmaier, “Heuristic decision making,” Annual review of psychology, vol. 62, pp. 451–482, 2011.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Başak Oral (1 paper)
  2. Pierre Dragicevic (8 papers)
  3. Alexandru Telea (6 papers)
  4. Evanthia Dimara (5 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now