Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court (2403.13004v1)

Published 13 Mar 2024 in cs.CY, cs.AI, and cs.HC

Abstract: Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) -- automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors' non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (94)
  1. 2017. Automated License Plate Readers (ALPRs). https://www.eff.org/pages/automated-license-plate-readers-alpr. Accessed: 2022-09-13.
  2. 2021. Admissible evidence. Wex, by the Legal Information Institute at Cornell University. Accessed September 13, 2023. "https://www.law.cornell.edu/wex/admissible_evidence)."
  3. 2022. STRmix Use in Certified U.S. Forensic Labs Continues to Grow. https://www.strmix.com/news/strmix-use-in-certified-u-s-forensic-labs-continues-to-grow/. Accessed: 2023-09-13.
  4. 2023. After Third Wrongful Arrest, ACLU Slams Detroit Police Department for Continuing to Use Faulty Facial Recognition Technology. https://www.aclu.org/press-releases/after-third-wrongful-arrest-aclu-slams-detroit-police-department-for-continuing-to-use-faulty-facial-recognition-technology. Accessed: 2022-09-13.
  5. 2023. Motion. Wex, by the Legal Information Institute at Cornell University. Accessed September 13, 2023. "https://www.law.cornell.edu/wex/motion"
  6. Adversarial scrutiny of evidentiary statistical software. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1733–1746.
  7. Contestable Camera Cars: A Speculative Design Exploration of Public AI That Is Open and Responsive to Dispute. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
  8. Contestable AI by Design: Towards a Framework. Minds and Machines (2022), 1–27.
  9. Ali Alkhatib. 2021. To live in their utopia: Why algorithmic systems create absurd outcomes. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–9.
  10. Marco Almada. 2019. Human intervention in automated decision-making: Toward the construction of contestable systems. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law. 2–11.
  11. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI conference on human computation and crowdsourcing, Vol. 7. 2–11.
  12. Designing disaggregated evaluations of ai systems: Choices, considerations, and tradeoffs. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 368–378.
  13. Hugh Beyer and Karen Holtzblatt. 1999. Contextual design. interactions 6, 1 (1999), 32–42.
  14. Power to the people? opportunities and challenges for participatory AI. Equity and Access in Algorithms, Mechanisms, and Optimization (2022), 1–8.
  15. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77–91.
  16. How AI-powered tech landed man in jail with scant evidence. https://apnews.com/article/artificial-intelligence-algorithm-technology-police-crime-7e3345485aa668c97606d4b54f9b6220. Accessed: 2022-09-13.
  17. DNA Mixture Interpretation: A NIST Scientific Foundation Review. National Institutes of Standards and Technology (2021).
  18. Zeno: An interactive framework for behavioral evaluation of machine learning. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–14.
  19. ” Hello AI”: uncovering the onboarding needs of medical practitioners for human-AI collaborative decision-making. Proceedings of the ACM on Human-computer Interaction 3, CSCW (2019), 1–24.
  20. Marc Canellas. 2021. Defending IEEE Software Standards in Federal Criminal Court. Computer 54, 6 (2021), 14–23.
  21. Kathy Charmaz. 2006. Constructing grounded theory: A practical guide through qualitative analysis. sage.
  22. Angèle Christin. 2017. Algorithms in practice: Comparing web journalism and criminal justice. Big Data & Society 4, 2 (2017), 2053951717718855.
  23. Michael D Coble and Jo-Anne Bright. 2019. Probabilistic genotyping software: an overview. Forensic Science International: Genetics 38 (2019), 219–224.
  24. Library of Congress Congress.gov. 2021. H.R.2438 - 117th Congress (2021-2022): Justice in Forensic Algorithms Act of 2021. Retrieved February 23, 2023 from https://www.congress.gov/bill/117th-congress/house-bill/2438
  25. Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 1571–1583.
  26. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 1–23.
  27. Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–18.
  28. Toward User-Driven Algorithm Auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19.
  29. The who in explainable ai: How ai background shapes perceptions of ai explanations. arXiv preprint arXiv:2107.13509 (2021).
  30. Donald J Farole and Lynn Langton. 2010. County-based and local public defender offices, 2007. US Department of Justice, Office of Justice Programs, Bureau of Justice ….
  31. Todd Feathers. 2021. Police are telling ShotSpotter to alter evidence from gunshot-detecting AI. Retrieved November 23 (2021), 2022.
  32. Bryan Furst. 2019. A Fair Fight. Brennan Center for Justice, September 9 (2019).
  33. Error rates, likelihood ratios, and jury evaluation of forensic evidence. Journal of Forensic Sciences 65, 4 (2020), 1199–1209.
  34. Clare Garvie. 2016. The perpetual line-up: Unregulated police face recognition in America. Georgetown Law, Center on Privacy & Technology.
  35. Clare Garvie. 2022. A forensic without the science: face recognition in US criminal investigations. Center on Privacy & Technology at Georgetown Law 6 (2022).
  36. Understanding the impact of police body-worn cameras on Virginia public defenders. Retrieved March 2 (2019), 2022.
  37. What Else Do I Need to Know? The Effect of Background Information on Users’ Reliance on AI Systems. arXiv preprint arXiv:2305.14331 (2023).
  38. Caroline Wolf Harlow. 2001. Defense counsel in criminal cases. US Department of Justice, Office of Justice Programs, Bureau of Justice ….
  39. Mireille Hildebrandt. 2019. Privacy as protection of the incomputable self: From agnostic to agonistic machine learning. Theoretical Inquiries in Law 20, 1 (2019), 83–121.
  40. Kashmir Hill. 2020. Wrongfully Accused by an Algorithm. https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html. Accessed: 2022-09-13.
  41. Designing contestability: Interaction design, machine learning, and mental health. In Proceedings of the 2017 Conference on Designing Interactive Systems. 95–99.
  42. Critical Evaluation Gaps in Machine Learning Practice. (2022).
  43. Elizabeth E Joh. 2017. The undue influence of surveillance technology companies in policing. NYUL Rev. Online 92 (2017), 19.
  44. J Jouvenal. 2021. A secret algorithm is transforming DNA evidence. This defendant could be the first to scrutinize it. Washington Post, July 13 (2021), 2021.
  45. Margot E Kaminski and Jennifer M Urban. 2021. The right to contest AI. Columbia Law Review 121, 7 (2021), 1957–2048.
  46. Improving human-AI partnerships in child welfare: understanding worker practices, challenges, and desires for algorithmic decision support. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18.
  47. “Why Do I Care What’s Similar?” Probing Challenges in AI-Assisted Child Welfare Decision-Making through Worker-AI Interface Design Concepts. In Designing Interactive Systems Conference. 454–470.
  48. Lauren Kirchner. 2017a. Thousands of Criminal Cases in New York Relied on Disputed DNA Testing Techniques. https://www.propublica.org/article/thousands-of-criminal-cases-in-new-york-relied-on-disputed-dna-testing-techniques. Accessed: 2022-09-13.
  49. Lauren Kirchner. 2017b. Thousands of Criminal Cases in New York Relied on Disputed DNA Testing Techniques. ProPublica (4 Sept. 2017) (2017).
  50. Shaping our tools: Contestability as a means to promote responsible algorithmic decision making in the professions. Ethics of Data and Analytics. Auerbach Publications (2022), 420–428.
  51. Defining AI in policy versus practice. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 72–78.
  52. Dan E Krane and M Katherine Philpott. 2022. Using Laboratory Validation to Identify and Establish Limits to the Reliability of Probabilistic Genotyping Systems. In Handbook of DNA Profiling. Springer, 297–319.
  53. Understanding Frontline Workers’ and Unhoused Individuals’ Perspectives on AI Used in Homeless Services. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
  54. Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency. 29–38.
  55. End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic Behavior. Proceedings of the ACM on Human-Computer Interaction 6, CSCW2 (2022), 1–34.
  56. The bureaucratic challenge to AI governance: An empirical assessment of implementation at US federal agencies. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. 606–652.
  57. Q Vera Liao and Ziang Xiao. 2023. Rethinking Model Evaluation as Narrowing the Socio-Technical Gap. arXiv preprint arXiv:2306.03100 (2023).
  58. Steven P Lund and Hari Iyer. 2017. Likelihood ratio as weight of forensic evidence: a closer look. Journal of Research of the National Institute of Standards and Technology 122 (2017), 1.
  59. Katie Lynch. [n. d.]. Know the facts. http://www.beingatticusfinch.com/know-the-facts
  60. What’s the appeal? Perceptions of review processes for algorithmic decisions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–15.
  61. The right to confront your accusers: Opening the black box of forensic DNA software. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 321–327.
  62. When trusted black boxes don’t agree: Incentivizing iterative improvement and accountability in critical software systems. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 102–108.
  63. Taking Algorithms to Courts: A Relational Approach to Algorithmic Accountability. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1450–1462.
  64. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.
  65. Automated fingerprint identification system (AFIS). The fingerprint sourcebook 1 (2011), 6–1.
  66. Contextual analysis of social media: The promise and challenge of eliciting context in social media posts with natural language processing. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 337–342.
  67. Application of machine learning to toolmarks: statistically based methods for impression pattern comparisons. National Institute of Justice (2012). Pierre et al. (2021) Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11. Ploug and Holm (2020) Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Jennifer Pierre, Roderic Crooks, Morgan Currie, Britt Paris, and Irene Pasquetto. 2021. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11. Ploug and Holm (2020) Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  68. Getting Ourselves Together: Data-centered participatory design research & epistemic burden. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–11. Ploug and Holm (2020) Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  69. Thomas Ploug and Søren Holm. 2020. The four dimensions of contestable AI diagnostics-A patient-centric approach to explainable AI. Artificial Intelligence in Medicine 107 (2020), 101901. Prabhudesai et al. (2023) Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Snehal Prabhudesai, Leyao Yang, Sumit Asthana, Xun Huan, Q Vera Liao, and Nikola Banovic. 2023. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  70. Understanding Uncertainty: How Lay Decision-makers Perceive and Interpret Uncertainty in Human-AI Decision Making. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 379–396. Raji et al. (2021) Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  71. AI and the everything in the whole wide world benchmark. arXiv preprint arXiv:2111.15366 (2021). Richardson (2021) Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  72. Rashida Richardson. 2021. Defining and demystifying automated decision systems. Md. L. Rev. 81 (2021), 785. Sarra (2020) Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  73. Claudio Sarra. 2020. Put dialectics into the machine: protection against automatic-decision-making through a deeper understanding of contestability by design. Global Jurist 20, 3 (2020), 20200003. Saxena et al. (2021) Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  74. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child-welfare. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–41. Shen et al. (2021) Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Hong Shen, Alicia DeVos, Motahhare Eslami, and Kenneth Holstein. 2021. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  75. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29. Siems et al. (2022) Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eli Siems, Katherine J Strandburg, and Nicholas Vincent. 2022. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  76. Trade Secrecy and Innovation in Forensic Technology. Hastings LJ 73 (2022), 773. Spangenberg and Beeman (1995) Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  77. Robert L Spangenberg and Marea L Beeman. 1995. Indigent defense systems in the United States. Law & Contemp. Probs. 58 (1995), 31. Suresh et al. (2021) Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Steven R Gomez, Kevin K Nam, and Arvind Satyanarayan. 2021. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  78. Beyond expertise and roles: A framework to characterize the stakeholders of interpretable machine learning and their needs. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16. Suresh et al. (2022) Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Rajiv Movva, Amelia Lee Dogan, Rahul Bhargava, Isadora Cruxên, Ángeles Martinez Cuba, Guilia Taurino, Wonyoung So, and Catherine D’Ignazio. 2022. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  79. Towards intersectional feminist and participatory ML: A case study in supporting Feminicide Counterdata Collection. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 667–678. Suresh et al. (2023) Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Harini Suresh, Divya Shanmugam, Tiffany Chen, Annie G Bryan, Alexander D’Amour, John Guttag, and Arvind Satyanarayan. 2023. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  80. Kaleidoscope: Semantically-grounded, context-specific ML model evaluation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13. Taori et al. (2020) Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  81. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems 33 (2020), 18583–18599. Vaccaro et al. (2019) Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Karrie Karahalios, Deirdre K Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  82. Contestability in algorithmic systems. In Conference companion publication of the 2019 on computer supported cooperative work and social computing. 523–527. Vaccaro et al. (2020) Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. 2020. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  83. ” At the End of the Day Facebook Does What ItWants” How Users Experience Contesting Algorithmic Content Moderation. Proceedings of the ACM on human-computer interaction 4, CSCW2 (2020), 1–22. Warren and Salehi (2022) Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  84. Rachel B Warren and Niloufar Salehi. 2022. Trial by File Formats: Exploring Public Defenders’ Challenges Working with Novel Surveillance Data. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26. Wexler (2017) Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  85. Rebecca Wexler. 2017. When a computer program keeps you in jail: How computers are harming criminal justice. New York Times 13 (2017). Wexler (2018) Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  86. Rebecca Wexler. 2018. Life, liberty, and trade secrets: Intellectual property in the criminal justice system. Stan. L. Rev. 70 (2018), 1343. Wilson-Kovacs et al. (2023) Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Dana Wilson-Kovacs, Rebecca Helm, Beth Growns, and Lauren Redfern. 2023. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  87. Digital evidence in defence practice: Prevalence, challenges and expertise. The International Journal of Evidence & Proof (2023), 13657127231171620. Wright et al. ([n. d.]) Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Lucas Wright, Roxana Mika Muenster, Briana Vecchione, Tianyao Qu, Senhuang (Pika) Cai, Alan Smith, Jacob Metcalf, and J. Nathan Matias. [n. d.]. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  88. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. ([n. d.]). https://doi.org/10.17605/OSF.IO/UPFDK Wu et al. (2021) Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Eric Wu, Kevin Wu, Roxana Daneshjou, David Ouyang, Daniel E Ho, and James Zou. 2021. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  89. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nature Medicine 27, 4 (2021), 582–584. Yang et al. (2023a) Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Fumeng Yang, Maryam Hedayati, and Matthew Kay. 2023a. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  90. Subjective Probability Correction for Uncertainty Representations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17. Yang et al. (2023b) Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Qian Yang, Richmond Y Wong, Thomas Gilbert, Margaret D Hagan, Steven Jackson, Sabine Junginger, and John Zimmerman. 2023b. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  91. Designing Technology and Policy Simultaneously: Towards A Research Agenda and New Practice. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–6. Yin et al. (2019) Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  92. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12. Yurrita et al. (2023) Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Mireia Yurrita, Tim Draws, Agathe Balayn, Dave Murray-Rust, Nava Tintarev, and Alessandro Bozzon. 2023. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  93. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21. Zhang et al. (2020) Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305. Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
  94. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 295–305.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now