Iceberg Sensemaking: A Process Model for Critical Data Analysis and Visualization (2204.04758v4)
Abstract: We offer a new model of the sensemaking process for data analysis and visualization. Whereas past sensemaking models have been grounded in positivist assumptions about the nature of knowledge, we reframe data sensemaking in critical, humanistic terms by approaching it through an interpretivist lens. Our three-phase process model uses the analogy of an iceberg, where data is the visible tip of underlying schemas. In the Add phase, the analyst acquires data, incorporates explicit schemas from the data, and absorbs the tacit schemas of both data and people. In the Check phase, the analyst interprets the data with respect to the current schemas and evaluates whether the schemas match the data. In the Refine phase, the analyst considers the role of power, articulates what was tacit into explicitly stated schemas, updates data, and formulates findings. Our model has four important distinguishing features: Tacit and Explicit Schemas, Schemas First and Always, Data as a Schematic Artifact, and Schematic Multiplicity. We compare the roles of schemas in past sensemaking models and draw conceptual distinctions based on a historical review of schemas in different academic traditions. We validate the descriptive and prescriptive power of our model through four analysis scenarios: noticing uncollected data, learning to wrangle data, downplaying inconvenient data, and measuring with sensors. We conclude by discussing the value of interpretivism, the virtue of epistemic humility, and the pluralism this sensemaking model can foster.
- S. Barocas and d. boyd, “Engaging the ethics of data science in practice,” Communications of the ACM, vol. 60, no. 11, pp. 23–25, 2017.
- d. boyd and K. Crawford, “Six provocations for big data,” Keynote Address: A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society, 2011.
- ——, “Critical questions for big data,” Information, Communication & Society, vol. 15, no. 5, pp. 662–679, 2012. [Online]. Available: https://doi.org/10.1080/1369118X.2012.678878
- M. Correll, “Ethical dimensions of visualization research,” in Proc. ACM Conf. Human Factors in Computing Systems (CHI), 2019, pp. 1–13.
- M. Dörk, P. Feng, C. Collins, and S. Carpendale, “Critical InfoVis: exploring the politics of visualization,” in Proc. ACM CHI Extended Abstracts on Human Factors in Computing Systems, 2013, p. 2189–2198.
- C. D’Ignazio and L. F. Klein, “Feminist data visualization,” in IEEE VIS Workshop on Visualization for the Digital Humanities (Vis4DH), 2016, p. 5.
- C. Cadwalladr and E. Graham-Harrison, “Revealed: 50 million facebook profiles harvested for cambridge analytica in major data breach,” The Guardian, 2018. [Online]. Available: https://www.theguardian.com/news/2018/mar/17/cambridge-analytica-facebook-influence-us-election
- A. Shapiro, “Reform predictive policing,” Nature, vol. 541, no. 7638, pp. 458–460, 2017.
- S. Brayne, “Big data surveillance: The case of policing,” American Sociological Review, vol. 82, no. 5, pp. 977–1008, 2017.
- J. Angwin, J. Larson, S. Mattu, and L. Kirchner, “Machine bias,” ProPublica, 2016-05-23.
- M. L. Jones, “How we became instrumentalists (again): Data positivism since world war II,” Historical Studies in the Natural Sciences, vol. 48, no. 5, 2021.
- J. Drucker, “Visualizing temporality and chronologies for the humanities,” IEEE VIS 2019 Capstone, 2019. [Online]. Available: https://vimeo.com/369216256
- M. Meyer and J. Dykes, “Criteria for rigor in visualization design study,” IEEE Transactions on Visualization and Computer Graphics, pp. 1–1, 2019.
- D. M. Russell, M. J. Stefik, P. Pirolli, and S. K. Card, “The cost structure of sensemaking,” in Proc Conf. Human Factors in Computing Systems (CHI), 1993, pp. 269–276.
- G. Grolemund and H. Wickham, “A cognitive interpretation of data analysis,” International Statistical Review, vol. 82, no. 2, pp. 184–204, 2014. [Online]. Available: http://doi.wiley.com/10.1111/insr.12028
- D. Sacha, A. Stoffel, F. Stoffel, B. C. Kwon, G. Ellis, and D. A. Keim, “Knowledge generation model for visual analytics,” IEEE Transactions on Visualization and Computer Graphics, vol. 20, no. 12, pp. 1604–1613, 2014.
- M. Meyer, M. Sedlmair, P. S. Quinan, and T. Munzner, “The nested blocks and guidelines model,” Information Visualization, vol. 14, no. 3, pp. 234–249, 2015. [Online]. Available: http://journals.sagepub.com/doi/10.1177/1473871613510429
- P. Pirolli and S. Card, “The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis,” in Proceedings of International Conference on Intelligence Analysis, 2005, p. 6.
- J. W. Tukey, “The future of data analysis,” The Annals of Mathematical Statistics, vol. 33, no. 1, pp. 1–67, 1962. [Online]. Available: http://www.jstor.org/stable/2237638
- ——, “Data analysis, computation, and mathematics,” Quarterly of applied mathematics, vol. 30, no. 1, pp. 51–65, 1972.
- J. Hullman and A. Gelman, “Designing for interactive exploratory data analysis requires theories of graphical inference,” Harvard Data Science Review, vol. 3, no. 3, 2021.
- N. Chomsky, “Three models for the description of language,” IRE Transactions on Information Theory, vol. 2, no. 3, pp. 113–124, 1956.
- A. Newell, J. C. Shaw, and H. A. Simon, “Report on a general problem-solving program,” in Proceedings of the International Conference on Information Processing (IFIP), 1959, pp. 256–264.
- A. Turing, “Computing machinery and intelligence,” Mind, 1937.
- C. E. Shannon, “A chess-playing machine,” Scientific American, vol. 182, no. 2, pp. 48–51, 1950.
- M. Minsky, “A framework for representing knowledge,” MIT AI Lab Memo No. 306, Tech. Rep., 1974.
- I. Goldstein and S. Papert, “Artificial intelligence, language, and the study of knowledge,” Cognitive Science, vol. 1, no. 1, 1977.
- D. Engelbart, “Augmenting the human intellect: A conceptual framework,” Office of Scientific Research, Tech. Rep., 1962.
- A. Kay, “A personal computer for children of all ages [dynabook],” ACM National Conference, 1972.
- B. Shneiderman, “The future of interactive systems and the emergence of direct manipulation,” Behaviour and Information Technology, vol. 1, no. 3, pp. 237–256, 1982.
- E. L. Hutchins, J. D. Hollan, and D. A. Norman, “Direct manipulation interfaces,” Human-Computer Interaction, vol. 1, no. 4, pp. 311–338, 1985.
- G. Robertson, S. K. Card, and J. D. Mackinlay, “The cognitive coprocessor architecture for interactive user interfaces,” in Proc. ACM SIGGRAPH Symp. User Interface Software and Technology (UIST), 1989, pp. 10–18.
- S. L. Franconeri, L. M. Padilla, P. Shah, J. M. Zacks, and J. Hullman, “The science of visual data communication: What works:,” Psychological Science in the Public Interest, 2021.
- ——, “Response to vera and simon’s situated action: A symbolic interpretation,” Cognitive Science, vol. 17, 1993.
- L. Winner, “Do artifacts have politics?” Daedalus, vol. 109, no. 1, 1980.
- T. J. Pinch and W. E. Bijker, “The social construction of facts and artefacts: Or how the sociology of science and the sociology of technology might benefit each other,” Social studies of science, vol. 14, no. 3, pp. 399–441, 1984.
- D. Haraway, “Situated knowledges: The science question in feminism and the privilege of partial perspective,” Feminist Studies, vol. 14, no. 3, pp. 575–599, 1988.
- A. Karakatsanis, “The punishment bureaucracy: How to think about criminal justice reform,” Yale Law Journal, vol. 128, 2019.
- M. Ananny and K. Crawford, “algorithmic accountability,” new media, p. 17, 2018.
- E. Gettier, “Is justified true belief knowledge?” Analysis, vol. 23, no. 6, 1963.
- M. Beaudouin-Lafon, “Designing Interaction, not Interfaces,” in Proceedings of the working conference on Advanced visual interfaces - AVI ’04. Gallipoli, Italy: ACM Press, 2004, p. 15.
- S. Lavigne, B. Clifton, and F. Tseing, “Predicting financial crime: Augmenting the predictive policing arsenal,” ArXiv abs/1704.07826, 2017.
- S. Kandel, J. Heer, C. Plaisant, and J. Kennedy, “Research directions in data wrangling: Visualizations and transformations for usable and credible data,” Information visualization, vol. 10, no. 4, p. 271, 2011.
- S. Kasica, C. Berret, and T. Munzner, “Table scraps: An actionable framework for multi-table datawrangling from an artifact study of computational journalism,” IEEE Trans. Visualization and Computer Graphics (Proc. InfoVis 2020), vol. 27, no. 2, pp. 957–966, 2021.
- danah boyd, “In the pursuit of knowledge, there be dragons,” IEEE VIS 2021 Keynote. [Online]. Available: https://zephoria.substack.com/p/harnessing-dragons
- A. Goodnough, M. Davey, and M. Smith, “When the water turned brown,” The New York Times, 2016.
- C. Lee, T. Yang, G. Inchoco, G. M. Jones, and A. Satyanarayan, “Viral Visualizations: How Coronavirus Skeptics Use Orthodox Data Practices to Promote Unorthodox Science Online,” in ACM Human Factors in Computing Systems (CHI), 2021. [Online]. Available: http://vis.csail.mit.edu/pubs/viral-visualizations
- H. Lin, D. Akbaba, M. Meyer, and A. Lex, “Data hunches: Incorporating personal knowledge into visualizations,” IEEE Trans. Visualization and Computer Graphics, vol. 29, no. 1, 2023.
- Big Data: A Tool for Inclusion or Exclusion? Understanding the Issues. Federal Trade Commission, 2016.
- M. Correll, “Counting, collaborating, and coexisting: Visualization and the digital humanities,” IEEE VIS Workshop on Visualization for the Digital Humanities (Vis4DH) Capstone, 2019.
- ——, “Ethical dimensions of visualization research,” In Proc. ACM Conf. Human Factors in Computing Systems (CHI), 2019.
- D. Haraway, “Situated knowledges: The science question in feminism and the privilege of partial perspective,” Feminist Studies, vol. 14, no. 3, 2018.
- J. Drucker, “Humanities approaches to graphical display,” Digital Humanities Quarterly (DHQ), vol. 5, no. 1, 2011. [Online]. Available: http://digitalhumanities.org/dhq/vol/5/1/000091/000091.html
- Charles Berret (2 papers)
- Tamara Munzner (14 papers)