Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Large Language Models in Law: A Survey (2312.03718v1)

Published 26 Nov 2023 in cs.CL and cs.AI

Abstract: The advent of AI has significantly impacted the traditional judicial industry. Moreover, recently, with the development of AI-generated content (AIGC), AI and law have found applications in various domains, including image recognition, automatic text generation, and interactive chat. With the rapid emergence and growing popularity of large models, it is evident that AI will drive transformation in the traditional judicial industry. However, the application of legal LLMs is still in its nascent stage. Several challenges need to be addressed. In this paper, we aim to provide a comprehensive survey of legal LLMs. We not only conduct an extensive survey of LLMs, but also expose their applications in the judicial system. We first provide an overview of AI technologies in the legal field and showcase the recent research in LLMs. Then, we discuss the practical implementation presented by legal LLMs, such as providing legal advice to users and assisting judges during trials. In addition, we explore the limitations of legal LLMs, including data, algorithms, and judicial practice. Finally, we summarize practical recommendations and propose future development directions to address these challenges.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (163)
  1. TensorFlow: a system for large-scale machine learning, in: The 12th USENIX Symposium on Operating Systems Design and Implementation, pp. 265–283.
  2. An analytical study of information extraction from unstructured and multidimensional big data. Journal of Big Data 6, 1–38.
  3. A summary of the research on the judicial application of artificial intelligence. Chinese Studies 9, 14.
  4. Using multi shares for ensuring privacy in database-as-a-service, in: The 44th Hawaii International Conference on System Sciences, IEEE. pp. 1–9.
  5. Explanation in AI and law: Past, present and future. Artificial Intelligence 289, 103387.
  6. Is ChatGPT leading generative AI? what is beyond expectations? Academic Platform Journal of Engineering and Smart Systems 11, 118–134.
  7. Precedent and discretion. The Supreme Court Review 2019, 313–334.
  8. A neural probabilistic language model. Journal of Machine Learning Research 3, 1137–1155.
  9. Does the use of risk assessments in sentences respect the right to due process? a critical analysis of the wisconsin v. loomis ruling. Law, Probability and Risk 17, 45–53.
  10. Containers and cloud: From LXC to docker to kubernetes. IEEE Cloud Computing 1, 81–84.
  11. On the opportunities and risks of foundation models. arXiv preprint, arXiv:2108.07258 .
  12. Large language models in machine translation. EMNLP-CoNLL , 858.
  13. Graphics processing unit (GPU) programming strategies and trends in gpu computing. Journal of Parallel and Distributed Computing 73, 4–13.
  14. Class-based n-gram models of natural language. Computational Linguistics 18, 467–480.
  15. Language models are few-shot learners. Advances in Neural Information Processing Systems 33, 1877–1901.
  16. Artificial intelligence, for real. Harvard Business Review 1, 1–31.
  17. A search engine for natural language applications, in: The 14th International Conference on World Wide Web, pp. 442–452.
  18. Genie: A generator of natural language semantic parsers for virtual assistant commands, in: The 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 394–410.
  19. AI in finance: challenges, techniques, and opportunities. ACM Computing Surveys 55, 1–38.
  20. A comprehensive survey of AI-generated content (AIGC): A history of generative AI from GAN to ChatGPT. arXiv preprint, arXiv:2303.04226 .
  21. Extracting training data from large language models, in: 30th USENIX Security Symposium, pp. 2633–2650.
  22. Deep learning in law: early adaptation and legal word embeddings trained on large corpora. Artificial Intelligence and Law 27, 171–198.
  23. A deep learning method for judicial decision support, in: IEEE 19th International Conference on Software Quality, Reliability and Security Companion, IEEE. pp. 145–149.
  24. Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing, in: IEEE 17th International Symposium on High Performance Computer Architecture, IEEE. pp. 266–277.
  25. Artificial intelligence in education: A review. IEEE Access 8, 75264–75278.
  26. Natural language processing. Fundamentals of Artificial Intelligence , 603–649.
  27. Word2vec. Natural Language Engineering 23, 155–162.
  28. Artificial intelligence and the transformation of humans, law and technology interactions in judicial proceedings. Law, Technology and Humans 2, 4–18.
  29. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis & Machine Intelligence , 1–20.
  30. ChatLaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint, arXiv:2306.16092 .
  31. Artificial intelligence and judicial modernization. Springer.
  32. Opportunities and challenges in explainable artificial intelligence (XAI): A survey. arXiv preprint, arXiv:2006.11371 .
  33. Data privacy: Definitions and techniques. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 20, 793–817.
  34. The judicial demand for explainable artificial intelligence. Columbia Law Review 119, 1829–1850.
  35. The matrix in context: Taking stock of police gang databases in london and beyond. Youth Justice 20, 11–30.
  36. Explainable artificial intelligence: A survey, in: The 41st International Convention on Information and Communication Technology, Electronics and Microelectronics, IEEE. pp. 0210–0215.
  37. Shortcut learning of large language models in natural language understanding: A survey. arXiv preprint, arXiv:2208.11857 .
  38. GLaM: Efficient scaling of language models with mixture-of-experts, in: International Conference on Machine Learning, PMLR. pp. 5547–5569.
  39. Stability and reliability in judicial decisions. Cornell Law Review 73, 422.
  40. Predictive policing: not yet, but soon preemptive? Policing and Society 30, 905–919.
  41. Applications of artificial intelligence in agriculture: A review. Engineering, Technology & Applied Science Research 9.
  42. Investigating the listening and transcription performance in court: experiences from stenographers in philippine courtrooms. Journal of Language and Pragmatics Studies 2, 100–111.
  43. The impact of artificial intelligence on rules, standards, and judicial discretion. Southern California Law Review 93, 1.
  44. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. The Journal of Machine Learning Research 23, 5232–5270.
  45. Minds, bodies, and machines. Artificial Intelligence: Its Scope and Limits , 269–303.
  46. False positives, false negatives, and false analyses: A rejoinder to machine bias: There’s software used across the country to predict future criminals. and it’s biased against blacks. Federal Probation 80, 38.
  47. Large language models in education: Vision and opportunities, in: IEEE International Conference on Big Data, IEEE. pp. 1–10.
  48. Model-as-a-service (MaaS): A survey, in: IEEE International Conference on Big Data, IEEE. pp. 1–10.
  49. Deep learning. MIT Press.
  50. Building sustainable free legal advisory systems: Experiences from the history of AI & law. Computer Law & Security Review 34, 314–326.
  51. Preserving the rule of law in the era of artificial intelligence (AI). Artificial Intelligence and Law 30, 291–323.
  52. An introduction to neural networks. CRC Press.
  53. Concepts in law. volume 88. Springer Science & Business Media.
  54. Artificial intelligence in medicine. Metabolism 69, S36–S40.
  55. Transformer in transformer. Advances in Neural Information Processing Systems 34, 15908–15919.
  56. A court of specialists: Judicial behavior on the UK Supreme Court. Oxford University Press, USA.
  57. Predictive policing as a new tool for law enforcement? recent developments and challenges. European Journal on Criminal Policy and Research 24, 201–218.
  58. Learning distributed representations of concepts, in: The Eighth Annual Conference of the Cognitive Science Society, Amherst, MA. p. 12.
  59. Training compute-optimal large language models. arXiv preprint, arXiv:2203.15556 .
  60. Alexa, siri, cortana, and more: an introduction to voice assistants. Medical Reference Services Quarterly 37, 81–88.
  61. Lawyer LLaMA technical report. arXiv preprint, arXiv:2305.15062 .
  62. Caffe: Convolutional architecture for fast feature embedding, in: The 22nd ACM International Conference on Multimedia, pp. 675–678.
  63. Machine learning: Trends, perspectives, and prospects. Science 349, 255–260.
  64. A review on explainability in multimodal deep neural nets. IEEE Access 9, 59800–59821.
  65. In-datacenter performance analysis of a tensor processing unit, in: The 44th Annual International Symposium on Computer Architecture, pp. 1–12.
  66. Generalized optimal matching methods for causal inference. The Journal of Machine Learning Research 21, 2300–2353.
  67. Text summarization from legal documents: a survey. Artificial Intelligence Review 51, 371–402.
  68. ChatGPT for good? on opportunities and challenges of large language models for education. Learning and Individual Differences 103, 102274.
  69. Deep reinforcement learning for sequence-to-sequence models. IEEE Transactions on Neural Networks and Learning Systems 31, 2469–2489.
  70. BERT: Pre-training of deep bidirectional transformers for language understanding, in: NAACL-HLT, pp. 4171–4186.
  71. Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home), in: IEEE 8th Annual Computing and Communication Workshop and Conference, IEEE. pp. 99–103.
  72. A performance comparison of container-based technologies for the cloud. Future Generation Computer Systems 68, 175–182.
  73. Legal remedies for a forgiving society: Children’s rights, data protection rights and the value of forgiveness in AI-mediated risk profiling of children by dutch authorities. Computer Law & Security Review 38, 105430.
  74. A review and state of art of internet of things (IoT). Archives of Computational Methods in Engineering , 1–19.
  75. Deep learning. Nature 521, 436–444.
  76. PyTorch distributed: Experiences on accelerating data parallel training. The VLDB Endowment 13, 3005–3018.
  77. Towards understanding and mitigating social biases in language models, in: International Conference on Machine Learning, PMLR. pp. 6565–6576.
  78. When machine learning meets privacy: A survey and outlook. ACM Computing Surveys 54, 1–36.
  79. Deep learning for procedural content generation. Neural Computing and Applications 33, 19–37.
  80. Summary of ChatGPT-related research and perspective towards the future of large language models. Meta-Radiology , 100017.
  81. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint, arXiv:1907.11692 .
  82. BaGuaLu: targeting brain scale pretrained models with over 37 million cores, in: The 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 192–204.
  83. Predicting risk in criminal procedure: actuarial tools, algorithms, AI and judicial decision-making. Current Issues in Criminal Justice 32, 22–39.
  84. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys 56, 1–40.
  85. Natural language processing: an introduction. Journal of the American Medical Informatics Association 18, 544–551.
  86. A brief report on LawGPT 1.0: A virtual legal assistant based on GPT-3. arXiv preprint, arXiv:2302.05729 .
  87. Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models, in: Findings of the Association for Computational Linguistics, pp. 1864–1874.
  88. A review on the attention mechanism of deep learning. Neurocomputing 452, 48–62.
  89. AI in judicial application of law and the right to a court. Procedia Computer Science 192, 2220–2228.
  90. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35, 27730–27744.
  91. Recent progress on generative adversarial networks (gans): A survey. IEEE Access 7, 36322–36333.
  92. Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture, in: IEEE Intelligent Vehicles Symposium, IEEE. pp. 1672–1678.
  93. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32.
  94. Meaningful explanations of black box AI decision systems, in: The AAAI Conference on Artificial Intelligence, pp. 9780–9784.
  95. A comparison of sequence-to-sequence models for speech recognition., in: Interspeech, pp. 939–943.
  96. Improving access to justice in state courts with platform technology. Vanderbilt Law Review 70, 1993.
  97. Improving language understanding by generative pre-training .
  98. Language models are unsupervised multitask learners. OpenAI Blog 1, 9.
  99. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21, 5485–5551.
  100. Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science 48, 137–141.
  101. Artificial intelligence & human rights: Opportunities & risks. Berkman Klein Center Research Publication .
  102. Developing artificially intelligent justice. Stanford Technology Law Review 22, 242.
  103. Speech to text conversion using android platform. International Journal of Engineering Research and Application 3, 253–258.
  104. An automated conversation system using natural language processing (NLP) chatbot in python. Central Asian Journal of Medical and Natural Science 3, 314–336.
  105. AI and law: A fruitful synergy. Artificial Intelligence 150, 1–15.
  106. “that’s (not) the output i expected!” on the role of end user expectations in creating explanations of AI systems. Artificial Intelligence 298, 103507.
  107. Legal and human rights issues of AI: Gaps, challenges and vulnerabilities. Journal of Responsible Technology 4, 100005.
  108. Predictive crime mapping: Arbitrary grids or street networks? Journal of quantitative criminology 33, 569–594.
  109. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22500–22510.
  110. Artificial intelligence a modern approach. Pearson Education, Inc.
  111. The IBM 2015 english conversational telephone speech recognition system, in: Annual Conference of the International Speech Communication Association, pp. 3140–3144.
  112. Basic principles of term formation in the multilingual and multicultural context of EU law, in: Language and Culture in EU Law. Routledge, pp. 183–206.
  113. What language model to train if you have one million GPU hours? arXiv preprint, arXiv:2210.15424 .
  114. Towards a standard for identifying and managing bias in artificial intelligence. NIST Special Publication 1270.
  115. Intern: A new learning paradigm towards general vision. arXiv preprint arXiv:2111.08687 .
  116. Self-attention with relative position representations, in: Proceedings of NAACL-HLT, pp. 464–468.
  117. The smart court-a new pathway to justice in china?, in: International Journal for Court Administration, HeinOnline. p. 1.
  118. Compact graph architecture for speech emotion recognition, in: IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE. pp. 6284–6288.
  119. Megatron-LM: Training multi-billion parameter language models using model parallelism. arXiv preprint, arXiv:1909.08053 .
  120. A survey on image data augmentation for deep learning. Journal of big data 6, 1–48.
  121. Mastering the game of go without human knowledge. Nature 550, 354–359.
  122. Automated extraction of semantic legal metadata using natural language processing, in: IEEE 26th International Requirements Engineering Conference, IEEE. pp. 124–135.
  123. Using DeepSpeed and Megatron to train megatron-turing NLG 530b, a large-scale generative language model. arXiv preprint, arXiv:2201.11990 .
  124. Process for adapting language models to society (PALMS) with values-targeted datasets. Advances in Neural Information Processing Systems 34, 5861–5873.
  125. Judge v robot?: Artificial intelligence and judicial decision-making. University of New South Wales Law Journal, The 41, 1114–1133.
  126. Artificial intelligence and speedy trial in the judiciary: Myth, reality or need? a case study in the brazilian supreme court (STF). Government Information Quarterly 39, 101660.
  127. Can online courts promote access to justice? a case study of the internet courts in china. Computer Law & Security Review 39, 105461.
  128. Artificial intelligence and law: An overview. Georgia State University Law Review 35, 19–22.
  129. Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 27.
  130. Data sharing by scientists: practices and perceptions. PloS One 6, e21101.
  131. LLaMA: Open and efficient foundation language models. arXiv preprint, arXiv:2302.13971 .
  132. Speech to text and text to speech recognition systems-areview. IOSR Journal of Computer Engineering 20, 36–43.
  133. UGC-VQA: Benchmarking blind video quality assessment for user generated content. IEEE Transactions on Image Processing 30, 4449–4464.
  134. Attention is all you need. Advances in Neural Information Processing Systems 30.
  135. Sequence to sequence-video to text, in: IEEE International Conference on Computer Vision, pp. 4534–4542.
  136. The bottom-up evolution of representations in the transformer: A study with machine translation and language modeling objectives, in: EMNLP-IJCNLP, pp. 4396–4406.
  137. Why fairness cannot be automated: Bridging the gap between EU non-discrimination law and AI. Computer Law & Security Review 41, 105567.
  138. Emergent abilities of large language models. arXiv preprint, arXiv:2206.07682 .
  139. Federated learning with differential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Security 15, 3454–3469.
  140. Innovative research on legal talents training model in the era of artificial intelligence, in: 16th International Conference on Computer Science & Education, IEEE. pp. 257–262.
  141. Privacy asymmetries: Access to data in criminal defense investigations. UCLA Law Review 68, 212.
  142. AI-generated content (AIGC): A survey. arXiv preprint, arXiv:2304.06632 .
  143. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint, arXiv:1609.08144 .
  144. Lawformer: A pre-trained language model for chinese legal long documents. AI Open 2, 79–84.
  145. Human judges in the era of artificial intelligence: challenges and opportunities. Applied Artificial Intelligence 36, 2013652.
  146. LegalGNN: Legal information enhanced graph neural network for recommendation. ACM Transactions on Information Systems 40, 1–29.
  147. Transformers from an optimization perspective. Advances in Neural Information Processing Systems 35, 36958–36971.
  148. What’s inside the black box? AI challenges for lawyers and researchers. Legal Information Management 19, 2–13.
  149. Criminal justice, artificial intelligence systems, and human rights, in: ERA Forum, Springer. pp. 567–583.
  150. Large language models for robotics: A survey. arXiv preprint, arXiv:2311.07226 .
  151. Distributed training of large language models, in: The 29th IEEE International Conference on Parallel and Distributed Systems, IEEE. pp. 1–8.
  152. Pangu−--α𝛼\alphaitalic_α: Large-scale autoregressive pretrained chinese language models with auto-parallel computation. preprint arXiv:2104.12369 .
  153. Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration 23, 100224.
  154. Graph convolutional networks: a comprehensive review. Computational Social Networks 6, 1–23.
  155. Intelligent analysis and application of judicial big data sharing based on blockchain, in: 6th International Conference on Artificial Intelligence and Big Data, IEEE. pp. 592–596.
  156. Understanding bag-of-words model: a statistical framework. International Journal of Machine Learning and Cybernetics 1, 43–52.
  157. DIALOGPT: Large-scale generative pre-training for conversational response generation, in: The 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 270–278.
  158. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering 34, 5586–5609.
  159. A survey of large language models. arXiv preprint, arXiv:2303.18223 .
  160. Iteratively questioning and answering for interpretable legal judgment prediction, in: The AAAI Conference on Artificial Intelligence, pp. 1250–1257.
  161. How does NLP benefit legal system: A summary of legal artificial intelligence. arXiv preprint, arXiv:2004.12158 .
  162. JEC-QA: a legal-domain question answering dataset, in: The AAAI Conference on Artificial Intelligence, pp. 9701–9708.
  163. Strengthening legal protection against discrimination by algorithms and artificial intelligence. The International Journal of Human Rights 24, 1572–1593.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Jinqi Lai (1 paper)
  2. Wensheng Gan (80 papers)
  3. Jiayang Wu (64 papers)
  4. Zhenlian Qi (10 papers)
  5. Philip S. Yu (592 papers)
Citations (40)
View on Semantic Scholar