2000 character limit reached
Software Engineering Methods For AI-Driven Deductive Legal Reasoning (2404.09868v2)
Published 15 Apr 2024 in cs.SE, cs.CL, and cs.CY
Abstract: The recent proliferation of generative AI technologies such as pre-trained LLMs has opened up new frontiers in computational law. An exciting area of development is the use of AI to automate the deductive rule-based reasoning inherent in statutory and contract law. This paper argues that such automated deductive legal reasoning can now be viewed from the lens of software engineering, treating LLMs as interpreters of natural-language programs with natural-language inputs. We show how it is possible to apply principled software engineering techniques to enhance AI-driven legal reasoning of complex statutes and to unlock new applications in automated meta-reasoning such as mutation-guided example generation and metamorphic property-based testing.
- Robert S Arnold. Software change impact analysis. IEEE Computer Society Press, 1996.
- Kent Beck. Test-driven Development: By Example. Addison-Wesley Professional, 2003.
- Shelter Check: Proactively finding tax minimization strategies via AI. Tax Notes Federal, Dec, 12, 2022.
- Can GPT-3 perform statutory reasoning? arXiv preprint arXiv:2302.06100, 2023.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
- Metamorphic testing: A new approach for generating next test cases. Technical Report HKUST-CS98-01, The Hong Kong University of Science and Technology, 1998.
- Quickcheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the 5th ACM SIGPLAN International Conference on Functional Programming, ICFP, 2000.
- Jack W. Davidson and AM Hollersnm. Subprogram inlining: A study of its effects on program execution time. IEEE Transactions on Software Engineering, 18(2):89, 1992.
- Hints on test data selection: Help for the practicing programmer. Computer, 11(4):34–41, 1978.
- Damien Falato. Is it a trade or business? or a hobby or investment? https://www.thetaxadviser.com/newsletters/2023/may/is-it-a-trade-or-business-or-a-hobby-or-investment.html, May 2013. Online; Retreived March 9, 2024.
- Summary of the competition on legal information, extraction/entailment (COLIEE) 2023. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pages 472–480, 2023.
- Computable Contracts and Insurance: An Introduction. MIT Computational Law Report, August 2022. https://law.mit.edu/pub/computablecontractsandinsuranceanintroduction.
- A dataset for statutory reasoning in tax law entailment and question answering. arXiv preprint arXiv:2005.05257, 2020.
- Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213, 2022.
- Sarah B Lawsky. A logic for statutes. Fla. Tax Rev., 21:60, 2017.
- Automatic program repair. IEEE Software, 38(4):22–27, 2021.
- Hypothesis: A new approach to property-based testing. Journal of Open Source Software, 4(43):1891, 2019.
- Catala: a programming language for the law. Proceedings of the ACM on Programming Languages, 5(ICFP):1–29, 2021.
- Large language models as tax attorneys: a case study in legal capabilities emergence. Philosophical Transactions of the Royal Society A, 382(2270):20230159, 2024.
- Are mutation scores correlated with real fault detection? a large scale empirical study on the relationship between mutants and real faults. In Proceedings of the 40th international conference on software engineering, pages 537–548, 2018.
- Internal Revenue Service. Publication 525—taxable and nontaxable income. Department of the Treasury—IRS Publications, 2023.
- BERT-based ensemble methods with data augmentation for legal textual entailment in COLIEE statute law task. In Proceedings of the eighteenth international conference on artificial intelligence and law, pages 278–284, 2021.
- Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28(2):183–200, 2002.