Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

'Put the Car on the Stand': SMT-based Oracles for Investigating Decisions (2305.05731v2)

Published 9 May 2023 in cs.LO, cs.CY, and cs.PL

Abstract: Principled accountability in the aftermath of harms is essential to the trustworthy design and governance of algorithmic decision making. Legal theory offers a paramount method for assessing culpability: putting the agent 'on the stand' to subject their actions and intentions to cross-examination. We show that under minimal assumptions automated reasoning can rigorously interrogate algorithmic behaviors as in the adversarial process of legal fact finding. We model accountability processes, such as trials or review boards, as Counterfactual-Guided Logic Exploration and Abstraction Refinement (CLEAR) loops. We use the formal methods of symbolic execution and satisfiability modulo theories (SMT) solving to discharge queries about agent behavior in factual and counterfactual scenarios, as adaptively formulated by a human investigator. In order to do so, for a decision algorithm $\mathcal{A}$ we use symbolic execution to represent its logic as a statement $\Pi$ in the decidable theory $\texttt{QF_FPBV}$. We implement our framework and demonstrate its utility on an illustrative car crash scenario.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (73)
  1. Adversarial Scrutiny of Evidentiary Statistical Software. In ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22), pages 1733–1746, 2022.
  2. Peeking Inside the Black-Box: a Survey on Explainable Artificial Intelligence (XAI). IEEE Access, 6:52138–52160, 2018.
  3. Safe Reinforcement Learning via Shielding. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI ’18), volume 32, 2018.
  4. Machine Bias. ProPublica, May 23rd, 2016. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
  5. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Information Fusion, 58:82–115, 2020.
  6. From Verification to Causality-based Explications. arXiv preprint arXiv:2105.09533, 2021.
  7. A Game-Theoretic Account of Responsibility Allocation. arXiv preprint arXiv:2105.09129, 2021.
  8. Responsibility Attribution in Parameterized Markovian Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11734–11743, 2021.
  9. A Survey of Symbolic Execution Techniques. ACM Comput. Surv., 51(3), 2018.
  10. The SMT-LIB Standard: Version 2.6, 2021.
  11. Counterfactual Theories of Causation. In Edward N. Zalta, editor, Stanford Encyclopedia of Philosophy. Stanford University, 2019.
  12. Symbolic Execution with CEGAR. In International Symposium on Leveraging Applications of Formal Methods, Verification and Validation (ISoLA ’16), pages 195–211. Springer, 2016.
  13. Michael Bratman. Intention, Plans, and Practical Reason, 1987.
  14. Plans and resource-bounded practical reasoning. Computational intelligence, 4(3):349–355, 1988.
  15. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In USENIX Symposium on Operating Systems Design and Implementation (OSDI ’08), pages 209–224, 2008.
  16. Analyzing intentional behavior in autonomous agents under uncertainty. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23, pages 372–381, 2023.
  17. Duckietown Environments for OpenAI Gym. https://github.com/duckietown/gym-duckietown, 2018.
  18. Responsibility and Blame: A Structural-Model Approach. Journal of Artificial Intelligence Research), 22:93–115, 2004.
  19. Automated Safety Verification of Programs Invoking Neural Networks. In International Conference on Computer Aided Verification (CAV ’21), pages 201–224. Springer, 2021.
  20. Counterexample-Guided Abstraction Refinement. In International Conference on Computer Aided Verification (CAV ’00), pages 154–169. Springer, 2000.
  21. Intention is Choice with Commitment. Artificial Intelligence, 42(2-3):213–261, 1990.
  22. David Cole. The Chinese Room Argument. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, 2020.
  23. How They Did It: An Analysis of Emission Defeat Devices in Modern Automobiles. In IEEE Symposium on Security and Privacy (Oakland ’17), pages 231–250. IEEE, 2017.
  24. Continental AG. Continental Develops Intelligent Solutions for Automated Driving in the City, 2022. https://www.continental.com/en/press/press-releases/20220622-project-city/.
  25. Jeffrey Dastin. Amazon scraps secret AI recruiting tool that showed bias against women. Reuters, 2018. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G.
  26. Program Actions as Actual Causes: A Building Block for Accountability. In 2015 IEEE 28th Computer Security Foundations Symposium (CSF ’15), pages 261–275. IEEE, 2015.
  27. Model-based Theory Combination. Electronic Notes in Theoretical Computer Science, 198(2):37–49, 2008.
  28. Turning Catala into a Proof Platform for the Law. In Programming Languages and the Law (ProLaLa@POPL ’22), 2022.
  29. Tim W. Dornis. Artificial Creativity: Emergent Works and the Void in Current Copyright Doctrine. Yale JL & Tech., 22:1, 2020.
  30. VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-based Systems. In Intentional Conference on Computer Aided Verification (CAV ’19), pages 432–442. Springer, 2019.
  31. James Edwards. Theories of Criminal Law. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, 2021.
  32. Towards a Formal Model of Accountability. In Proceedings of the 2011 New Security Paradigms Workshop, pages 45–56, 2011.
  33. Accountability in Computing: Concepts and Mechanisms. Foundations and Trends® in Privacy and Security, 2(4):247–399, 2020.
  34. Scenic: A Language for Scenario Specification and Scene Generation. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’19), pages 63–78, 2019.
  35. A Comprehensive Survey on Safe Reinforcement Learning. Journal of Machine Learning Research, 16(1):1437–1480, 2015.
  36. AI2\text{AI}{{}^{2}}AI start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In 2018 IEEE Symposium on Security and Privacy (S&P ’18), pages 3–18, 2018.
  37. IMLI: An Incremental Framework for MaxSAT-based Learning of Interpretable Classification Rules. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’19), pages 203–210, 2019.
  38. A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys (CSUR), 51(5):1–42, 2018.
  39. Gabriel Hallevy. When Robots Kill: Artificial Intelligence Under Criminal Law. UPNE, 2013.
  40. Causes and Explanations: A Structural-Model Approach. Part I: Causes. The British Journal for the Philosophy of Science, 56(4):843–887, 2005.
  41. Causes and Explanations: A Structural-Model Approach. Part II: Explanations. The British Journal for the Philosophy of Science, 56(4):889–911, 2005.
  42. Software Model Checking. ACM Computing Surveys (CSUR), 41(4):1–54, 2009.
  43. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In International Conference on Computer Aided Verification (CAV ’17), pages 97–117, 2017.
  44. The Marabou Framework for Verification and Analysis of Deep Neural Networks. In International Conference on Computer Aided Verification (CAV ’19), pages 443–452, 2019.
  45. Formal Modeling and Analysis of Legal Contracts using ContractCheck. In Programming Languages and the Law (ProLaLa@POPL ’22), 2022.
  46. Accountable Algorithms. University of Pennsylvania Law Review, 165(3):633–705, 2017.
  47. Automated Verification of Accountability in Security Protocols. In 2019 IEEE 32nd Computer Security Foundations Symposium (CSF ’19), pages 397–39716. IEEE, 2019.
  48. Accountability: Definition and Relationship to Verifiability. In Proceedings of the 17th ACM Conference on Computer and Communications Security (CCS ’10), pages 526–535, 2010.
  49. Legal Information Institute. Respondeat Superior, 2023. https://www.law.cornell.edu/wex/respondeat_superior.
  50. Edward H Levi. An Introduction to Legal Reasoning. U. Chi. L. Rev., 15:501, 1947.
  51. David Lewis. Counterfactuals. John Wiley & Sons, 2013. Originally published in 1973.
  52. Floating-Point Symbolic Execution: A Case Study in N-Version Programming. In IEEE/ACM International Conference on Automated Software Engineering (ASE ’17), pages 601–612, 2017.
  53. Catala: A Programming Language for the Law. Proceedings of the ACM on Programming Languages, 5(ICFP):77:1–29, August 2021.
  54. Michael Moore. Causation in the Law. In Edward N. Zalta, editor, Stanford Encyclopedia of Philosophy. Stanford University, 2019.
  55. Explaining Machine Learning Classifiers Through Diverse Counterfactual Explanations. In ACM Conference on Fairness, Accountability, and Transparency (𝐹𝐴𝑇*superscript𝐹𝐴𝑇\text{FAT}^{*}FAT start_POSTSUPERSCRIPT * end_POSTSUPERSCRIPT ’20), pages 607–617, 2020.
  56. Z3: An Efficient SMT Solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS ’08), pages 337–340, 2008.
  57. Vincent C. Müller. Ethics of Artificial Intelligence and Robotics. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy. Stanford University, 2023.
  58. CirC: Compiler Infrastructure for Proof Systems, Software Verification, and more. In IEEE Symposium on Security and Privacy (Oakland ’22), pages 2248–2266, 2022.
  59. Black is the New Orange: How to Determine AI Liability. Artificial Intelligence and Law, 31(1):133–167, 2023.
  60. Pinocchio: Nearly practical verifiable computation. In 2013 IEEE Symposium on Security and Privacy, pages 238–252. IEEE, 2013.
  61. Modeling Rational Agents Within a BDI-Architecture. KR, 91:473–484, 1991.
  62. Proleg: An implementation of the presupposed ultimate fact theory of japanese civil code by prolog technology. In JSAI-isAI Workshops, volume 6797, pages 153–164. Springer, 2010.
  63. On a Formal Model of Safe and Scalable Self-Driving Cars. arXiv preprint arXiv:1708.06374, 2017.
  64. An Abstract Domain for Certifying Neural Networks. Proceedings of the ACM on Programming Languages, 3(POPL):1–30, 2019.
  65. Lauren Smiley. ‘I’m the Operator’: The Aftermath of a Self-Driving Tragedy. Wired Magazine, 2022. https://www.wired.com/story/uber-self-driving-car-fatal-crash/.
  66. William Starr. Counterfactuals. In Edward N. Zalta, editor, The Stanford Encyclopedia of Philosophy. Stanford University, 2021.
  67. Interpretable Two-Level Boolean Rule Learning for Classification. arXiv preprint arXiv:1511.07361, 2015.
  68. Symbolic Execution for Randomized Programs. Proceedings of the ACM on Programming Languages, 6(OOPSLA):1583–1612, 2022.
  69. Moshe Y. Vardi. Why is Modal Logic So Robustly Decidable? In Neil Immerman and Phokion G. Kolaitis, editors, Descriptive Complexity and Finite Models, Proceedings of a DIMACS Workshop 1996, volume 31, pages 149–183. DIMACS/AMS, 1996.
  70. Moshe Y. Vardi. Accountability and Liability in Computing. Communications of the ACM, 65(11):5, 2022.
  71. Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology, 31:841, 2017.
  72. Christopher J. C. H. Watkins and Peter Dayan. Q-learning. Machine Learning, 8:279–292, 1992.
  73. ADLP: Accountable Data Logging Protocol for Publish-Subscribe Communication Systems. In International Conference on Distributed Computing Systems (ICDCS ’19), pages 1149–1160. IEEE, 2019.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now