Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

What-if Analysis for Business Users: Current Practices and Future Opportunities (2212.13643v3)

Published 27 Dec 2022 in cs.HC and cs.DB

Abstract: What-if analysis (WIA), crucial for making data-driven decisions, enables users to understand how changes in variables impact outcomes and explore alternative scenarios. However, existing WIA research focuses on supporting the workflows of data scientists or analysts, largely overlooking significant non-technical users, like business users. We conduct a two-part user study with 22 business users (marketing, sales, product, and operations managers). The first study examines existing WIA techniques employed, tools used, and challenges faced. Findings reveal that business users perform many WIA techniques independently using rudimentary tools due to various constraints. We implement representative WIA techniques identified previously in a visual analytics prototype to use as a probe to conduct a follow-up study evaluating business users' practical use of the techniques. These techniques improve decision-making efficiency and confidence while highlighting the need for better support in data preparation, risk assessment, and domain knowledge integration. Finally, we offer design recommendations to enhance future business analytics systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer, “Enterprise data analysis and visualization: An interview study,” TVCG, vol. 18, no. 12, pp. 2917–2926, 2012.
  2. S. Alspaugh, N. Zokaei, A. Liu, C. Jin, and M. A. Hearst, “Futzing and moseying: Interviews with professional data analysts on exploration practices,” TVCG, vol. 25, no. 1, p. 22–31, jan 2019.
  3. E. Kandogan, A. Balakrishnan, E. M. Haber, and J. S. Pierce, “From data to insight: work practices of analysts in the enterprise,” CGA, vol. 34, no. 5, pp. 42–50, 2014.
  4. A. Crisan and B. Fiore-Gartland, “Fits and starts: Enterprise use of automl and the role of humans in the loop,” in CHI, 2021, pp. 1–15.
  5. P. Mikalef, M. N. Giannakos, I. O. Pappas, and J. Krogstie, “The human side of big data: Understanding the skills of the data scientist in education and industry,” in EDUCON 2018.   IEEE, 2018, pp. 503–512.
  6. Ç. Demiralp, P. J. Haas, S. Parthasarathy, and T. Pedapati, “Foresight: Rapid data exploration through guideposts,” CoRR, vol. abs/1709.10513, 2017.
  7. Report, “Advanced Analytics Report, 2022 - 2030.”
  8. ——, “Data Visualization Statistics 2023.”
  9. J. D. Little, “Decision support systems for marketing managers,” JM, vol. 43, no. 3, pp. 9–26, 1979.
  10. W. S. Perkins and R. C. Rao, “The role of experience in information use and decision making by marketing managers,” JMR, vol. 27, no. 1, pp. 1–10, 1990.
  11. F. Bergeron, L. Raymond, S. Rivard, and M.-F. Gara, “Determinants of eis use: Testing a behavioral model,” Decision Support Systems, vol. 14, no. 2, pp. 131–146, 1995.
  12. F. Marx, J. H. Mayer, and R. Winter, “Six principles for redesigning executive information systems—findings of a survey and evaluation of a prototype,” ACM TMIS, vol. 2, no. 4, pp. 1–19, 2012.
  13. G. H. Van Bruggen, B. Wierenga et al., “Marketing decision making and decision support: Challenges and perspectives for successful marketing management support systems,” Foundations and Trends in Marketing, vol. 4, no. 4, pp. 209–332, 2010.
  14. O. Velcu-Laitinen and O. M. Yigitbasioglu, “The use of dashboards in performance management: Evidence from sales managers.” IJDAR, vol. 12, 2012.
  15. E. Dimara, H. Zhang, M. Tory, and S. Franconeri, “The unmet data visualization needs of decision makers within organizations,” TVCG, 2021.
  16. M. Jasim, E. Hoque, A. Sarvghad, and N. Mahyar, “Communitypulse: Facilitating community input analysis by surfacing hidden insights, reflections, and priorities,” in DIS, 2021, pp. 846–863.
  17. L. South, M. Schwab, N. Beauchamp, L. Wang, J. Wihbey, and M. A. Borkin, “Debatevis: Visualizing political debates for non-expert users,” in VIS.   IEEE, 2020, pp. 241–245.
  18. Y. Kang and J. Stasko, “Examining the use of a visual analytics system for sensemaking tasks: Case studies with domain experts,” TVCG, vol. 18, no. 12, pp. 2869–2878, 2012.
  19. J. Liu, N. Boukhelifa, and J. Eagan, “Making sense of data workers’ sense making practices,” in CHI Workshop, 2019, p. 4.
  20. L. R. Bartram, M. Correll, and M. K. Tory, “Untidy data: The unreasonable effectiveness of tables,” TVCG, vol. PP, pp. 1–1, 2021.
  21. H. A. Simon, “The new science of management decision.” 1960.
  22. A. Berisha-Shaqiri, “Management information system and decision-making,” AJIS, vol. 3, no. 2, p. 19, 2014.
  23. S. Gathani, M. Hulsebos, J. Gale, P. J. Haas, and Ç. Demiralp, “Augmenting decision making via interactive what-if analysis,” in CIDR 2022. [Online]. Available: https://www.cidrdb.org/cidr2022/papers/p49-gathani.pdf
  24. Y. Zhang, Q. V. Liao, and R. K. Bellamy, “Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making,” in CFAT, 2020, pp. 295–305.
  25. Z. Zhang, S. Malik, S. Guo, J. Hoffswell, R. Rossi, F. Du, and E. Koh, “CODAS: Integrating Business Analytics and Report Authoring,” in EuroVA, J. Bernard and M. Angelini, Eds., 2022.
  26. G. Klein, B. Moon, and R. R. Hoffman, “Making sense of sensemaking 2: A macrocognitive model,” IEEE Intelligent Systems, vol. 21, no. 5, pp. 88–92, 2006.
  27. D. M. Russell, M. J. Stefik, P. Pirolli, and S. K. Card, “The cost structure of sensemaking,” in INTERACT and CHI, 1993, pp. 269–276.
  28. G. Klein, J. K. Phillips, E. L. Rall, and D. A. Peluso, “A data-frame theory of sensemaking,” in Expertise out of context: Proceedings of the sixth international conference on naturalistic decision making, vol. 113, 2007.
  29. P. Pirolli and S. Card, “The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis,” in CIA, vol. 5, 2005, pp. 2–4.
  30. F. C. Lunenburg, “The decision making process.” in National Forum of Educational Administration & Supervision Journal, vol. 27, no. 4, 2010.
  31. E. Bruch and F. Feinberg, “Decision-making processes in social contexts,” Annual review of sociology, vol. 43, pp. 207–227, 2017.
  32. C. Chen, “An information-theoretic view of visual analytics,” CGA, vol. 28, no. 1, pp. 18–23, 2008.
  33. N. E. Eric Newburger, “Visualization according to statisticians: An interview study on the role of visualization for inferential statistics,” 2023.
  34. W. Noonpakdee, T. Khunkornsiri, A. Phothichai, and K. Danaisawat, “A framework for analyzing and developing dashboard templates for small and medium enterprises,” in ICIEA.   IEEE, 2018, pp. 479–483.
  35. P. Bera, “How colors in business dashboards affect users’ decision making,” CACM, vol. 59, no. 4, pp. 50–57, 2016.
  36. E. Dimara and J. Stasko, “A critical reflection on visualization research: Where do decision making tasks hide?” TVCG, vol. 28, no. 1, pp. 1128–1138, 2021.
  37. A. Sarikaya, M. Correll, L. Bartram, M. Tory, and D. Fisher, “What do we talk about when we talk about dashboards?” TVCG, vol. 25, no. 1, pp. 682–692, 2018.
  38. M. Tory, L. Bartram, B. Fiore-Gartland, and A. Crisan, “Finding their data voice: Practices and challenges of dashboard users,” CGA, 2021.
  39. A. Crisan and M. Correll, “User ex machina: Simulation as a design probe in human-in-the-loop text analytics,” in CHI, 2021, pp. 1–16.
  40. D. Honeycutt, M. Nourani, and E. Ragan, “Soliciting human-in-the-loop user feedback for interactive machine learning reduces user trust and impressions of model accuracy,” in AAAI, vol. 8, 2020, pp. 63–72.
  41. D. Wang, J. D. Weisz, M. Muller, P. Ram, W. Geyer, C. Dugan, Y. Tausczik, H. Samulowitz, and A. Gray, “Human-ai collaboration in data science: Exploring data scientists’ perceptions of automated ai,” Proc. ACM Hum.-Comput. Interact., vol. 3, no. CSCW, nov 2019.
  42. F. Poursabzi-Sangdeh, D. G. Goldstein, J. M. Hofman, J. W. Wortman Vaughan, and H. Wallach, “Manipulating and measuring model interpretability,” in CHI, 2021, pp. 1–52.
  43. Tableau, “What is tableau business science?” 2021. [Online]. Available: https://www.tableau.com/about/blog/2021/3/what-is-tableau-business-science
  44. ——, “Einstein discovery,” 2021. [Online]. Available: https://www.tableau.com/products/add-ons/einstein-discovery
  45. SAS, “Sas visual analytics,” 2021. [Online]. Available: https://www.sas.com/en_us/software/visual-analytics.html
  46. Microsoft, “Solver: Microsoft Excel Documentation,” 2021.
  47. ——, “Goal Seek: Microsoft Excel Documentation,” 2021.
  48. E. Jun, “Empowering domain experts to author valid statistical analyses,” in UIST, 2022, pp. 1–5.
  49. S. Gathani, Z. Liu, P. J. Haas, and Ç. Demiralp, “Online Form for Participant Recruitment.”
  50. “User interviews,” 2021. [Online]. Available: https://www.userinterviews.com
  51. S. Gathani, Z. Liu, P. J. Haas, and Ç. Demiralp, “Supplementary Material.”
  52. Kaggle, “Kaggle Datasets.”
  53. A. Rappaport, “Sensitivity analysis in decision making,” The Accounting Review, vol. 42, no. 3, pp. 441–456, 1967.
  54. N. H. Lurie and C. H. Mason, “Visual representation: Implications for decision making,” JM, vol. 71, no. 1, pp. 160–177, 2007.
  55. S. Gathani, Z. Liu, P. J. Haas, and Ç. Demiralp, “Post Study Questionnaire.”
  56. H. Elhamdadi, A. Gaba, Y.-S. Kim, and C. Xiong, “How do we measure trust in visual data communication?” in BELIV, 2022, pp. 85–92.
  57. M. Langer, T. Hunsicker, T. Feldkamp, C. J. König, and N. Grgić-Hlača, ““look! it’sa computer program! it’s an algorithm! it’s ai!”: does terminology affect human perceptions and evaluations of algorithmic decision-making systems?” in CHI, 2022, pp. 1–28.
  58. H. Spitzeck and E. G. Hansen, “Stakeholder governance: how stakeholders influence corporate decision making,” Corporate Governance: The international journal of business in society, vol. 10, no. 4, pp. 378–391, 2010.
  59. A. Schreiber, L. von Kurnatowski, A. Meinecke, and C. de Boer, “An interactive dashboard for visualizing the provenance of software development processes,” in VISSOFT.   IEEE, 2021, pp. 100–104.
  60. H. S. Packer, A. Chapman, and L. Carr, “Github2prov: Provenance for supporting software project management.” in TaPP, 2019.
  61. Z. Cutler, K. Gadhave, and A. Lex, “Trrack: A library for provenance-tracking in web-based visualizations,” in VIS, 2020, pp. 116–120.
  62. K. Madanagopal, E. D. Ragan, and P. Benjamin, “Analytic provenance in practice: The role of provenance in real-world visualization and data analysis environments,” CGA, vol. 39, no. 6, pp. 30–45, 2019.
  63. E. D. Ragan, A. Endert, J. Sanyal, and J. Chen, “Characterizing provenance in visualization and data analysis: an organizational framework of provenance types and purposes,” TVCG, vol. 22, no. 1, pp. 31–40, 2015.
  64. M. H. Namaki, A. Floratou, F. Psallidas, S. Krishnan, A. Agrawal, Y. Wu, Y. Zhu, and M. Weimer, “Vamsa: Automated provenance tracking in data science scripts,” in SIGKDD, 2020, pp. 1542–1551.
  65. R. Souza, L. G. Azevedo, V. Lourenço, E. Soares, R. Thiago, R. Brandão, D. Civitarese, E. Vital Brazil, M. Moreno, P. Valduriez et al., “Workflow provenance in the lifecycle of scientific machine learning,” CCPE, vol. 34, no. 14, p. e6544, 2022.
  66. A. Kumar, R. McCann, J. Naughton, and J. M. Patel, “Model selection management systems: The next frontier of advanced analytics,” SIGMOD, vol. 44, no. 4, pp. 17–22, 2016.
  67. E. Jun, M. Daum, J. Roesch, S. Chasins, E. Berger, R. Just, and K. Reinecke, “Tea: A high-level language and runtime system for automating statistical analysis,” in UIST, 2019, pp. 591–603.
  68. E. Jun, M. Birchfield, N. De Moura, J. Heer, and R. Just, “Hypothesis formalization: Empirical findings, software limitations, and design implications,” TOCHI, vol. 29, no. 1, pp. 1–28, 2022.
  69. E. Jun, A. Seo, J. Heer, and R. Just, “Tisane: Authoring statistical models via formal reasoning from conceptual and data relationships,” in CHI, 2022, pp. 1–16.
  70. K. Gu, E. Jun, and T. Althoff, “Understanding and supporting debugging workflows in multiverse analysis,” in CHI, 2023, pp. 1–19.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now