Papers
Topics
Authors
Recent
Search
2000 character limit reached

Data Guards: Challenges and Solutions for Fostering Trust in Data

Published 19 Jul 2024 in cs.HC | (2407.14042v2)

Abstract: From dirty data to intentional deception, there are many threats to the validity of data-driven decisions. Making use of data, especially new or unfamiliar data, therefore requires a degree of trust or verification. How is this trust established? In this paper, we present the results of a series of interviews with both producers and consumers of data artifacts (outputs of data ecosystems like spreadsheets, charts, and dashboards) aimed at understanding strategies and obstacles to building trust in data. We find a recurring need, but lack of existing standards, for data validation and verification, especially among data consumers. We therefore propose a set of data guards: methods and tools for fostering trust in data artifacts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Dashboard design patterns. IEEE TVCG, 29(1):342–352, 2022.
  2. Untidy data: The unreasonable effectiveness of tables. IEEE TVCG, 28(1):686–696, 2021.
  3. Visual interactive creation, customization, and analysis of data quality metrics. Journal of Data and Information Quality (JDIQ), 10(1):1–26, 2018.
  4. M. Bouzeghoub. A framework for analysis of data freshness. In Proceedings of the 2004 international workshop on Information quality in information systems, pp. 59–67, 2004.
  5. V. Braun and V. Clarke. Thematic analysis., chap. 4, pp. 57–71. American Psychological Association, 2012.
  6. Vistrails: visualization meets data management. In Proceedings of ACM SIGMOD 2006, pp. 745–747, 2006.
  7. Data cleaning: Overview and emerging challenges. In Proceedings of ACM SIGMOD 2016, pp. 2201–2206, 2016.
  8. Passing the data baton: A retrospective analysis on data science work and workers. IEEE TVCG, 27(2):1860–1870, 2020.
  9. How do we measure trust in visual data communication? In 2022 IEEE Evaluation and Beyond-Methodological Approaches for Visualization (BELIV), pp. 85–92. IEEE, 2022.
  10. Annotating line charts for addressing deception. In Proceedings of ACM SIGCHI 2022, pp. 1–12, 2022.
  11. Hindsight: Encouraging exploration through direct encoding of personal interaction history. IEEE TVCG, 23(1):351–360, 2016.
  12. Data smells: Categories, causes and consequences, and detection of suspicious data in ai-based systems. In Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI, pp. 229–239, 2022.
  13. Datasheets for datasets. Communications of the ACM, 64(12):86–92, 2021.
  14. J. Heer. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences, 116(6):1844–1850, 2019.
  15. Does high public debt consistently stifle economic growth? a critique of reinhart and rogoff. Cambridge journal of economics, 38(2):257–279, 2014.
  16. H. M. Jannah. Metareader: A dataset meta-exploration and documentation tool, 2014.
  17. Enterprise data analysis and visualization: An interview study. IEEE TVCG, 18(12):2917–2926, 2012.
  18. Profiler: Integrated statistical analysis and visualization for data quality assessment. In Proceedings of ACM AVI 2012, pp. 547–554, 2012.
  19. From data to insight: work practices of analysts in the enterprise. IEEE computer graphics and applications, 34(5):42–50, 2014.
  20. A taxonomy of dirty data. Data mining and knowledge discovery, 7:81–99, 2003.
  21. Designing animated transitions to convey aggregate operations. Computer Graphics Forum, 38(3):541–551, 2019.
  22. Data through others’ eyes: The impact of visualizing others’ expectations on visualization interpretation. IEEE TVCG, 24(1):760–769, 2017.
  23. Ferret: Reviewing Tabular Datasets for Manipulation. Computer Graphics Forum, 42(3):187–198, June 2023.
  24. Misleading beyond visual tricks: How people actually lie with charts. In Proceedings of ACM SIGCHI 2023, pp. 1–21, 2023.
  25. Understanding the role of alternatives in data analysis practices. IEEE TVCG, 26(1):66–76, 2019.
  26. Steering data quality with visual analytics: The complexity challenge. Visual Informatics, 2(4):191–197, 2018.
  27. Surfacing visualization mirages. In Proceedings of ACM SIGCHI 2020, pp. 1–16, 2020.
  28. Model cards for model reporting. In Proceedings of ACM FAccT 2019, pp. 220–229, 2019.
  29. How data science workers work with data: Discovery, capture, curation, design, creation. In Proceedings of ACM SIGCHI 2019, pp. 1–15, 2019.
  30. F. Naumann. Data profiling revisited. ACM SIGMOD Record, 42(4):40–49, 2014.
  31. Distance matters. Human–computer interaction, 15(2-3):139–178, 2000.
  32. Data cards: Purposeful and transparent dataset documentation for responsible ai. In Proceedings of ACM FAccT 2022, pp. 1776–1826, 2022.
  33. Tasks and Visualizations Used for Data Profiling: A Survey and Interview Study. IEEE TVCG, pp. 1–12, 2024.
  34. The Role of Uncertainty, Awareness, and Trust in Visual Analytics. IEEE TVCG, 22(1):240–249, 2016.
  35. Odds and Insights: Decision Quality in Exploratory Data Analysis Under Uncertainty. preprint, Open Science Framework, 2024.
  36. Data smells in public datasets. In Proceedings of the 1st International Conference on AI Engineering: Software Engineering for AI, pp. 205–216, 2022.
  37. Using vistrails and provenance for teaching scientific visualization. Computer Graphics Forum, 30(1):75–84, 2011.
  38. Understanding the effects of visualizing missing values on visual data exploration. In 2021 IEEE Visualization Conference (VIS), pp. 161–165. IEEE, 2021.
  39. H. Song and D. A. Szafir. Where’s my data? evaluating visualizations with missing data. IEEE TVCG, 25(1):914–924, 2018.
  40. Finding their data voice: Practices and challenges of dashboard users. IEEE Computer Graphics and Applications, 43(1):22–36, 2021.
  41. Manyeyes: a site for visualization at internet scale. IEEE TVCG, 13(6):1121–1128, 2007.
  42. Beyond accuracy: What data quality means to data consumers. Journal of management information systems, 12(4):5–33, 1996.
  43. Effects of health and social care spending constraints on mortality in england: a time trend analysis. BMJ open, 7(11):e017722, 2017.
  44. I. Yanai and M. Lercher. A hypothesis is a liability. Genome Biology, 21(1):231, s13059–020–02133–w, Dec. 2020.
  45. Investigating the Effect of the Multiple Comparisons Problem in Visual Analysis. In Proceedings of ACM SIGCHI 2018, pp. 1–12. ACM, 2018.
  46. Safe visual data exploration. In Proceedings of ACM SIGMOD 2017, pp. 1671–1674, 2017.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free