Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction (2407.01964v4)

Published 2 Jul 2024 in cs.CL

Abstract: Legal judgment prediction is essential for enhancing judicial efficiency. In this work, we identify that existing LLMs underperform in this domain due to challenges in understanding case complexities and distinguishing between similar charges. To adapt LLMs for effective legal judgment prediction, we introduce the Ask-Discriminate-Predict (ADAPT) reasoning framework inspired by human judicial reasoning. ADAPT involves decomposing case facts, discriminating among potential charges, and predicting the final judgment. We further enhance LLMs through fine-tuning with multi-task synthetic trajectories to improve legal judgment prediction accuracy and efficiency under our ADAPT framework. Extensive experiments conducted on two widely-used datasets demonstrate the superior performance of our framework in legal judgment prediction, particularly when dealing with complex and confusing charges.

PDF Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. Qwen technical report. CoRR, abs/2309.16609.
  2. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
  3. LEGAL-BERT: "preparing the muppets for court’". In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pages 2898–2904. Association for Computational Linguistics.
  4. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171–4186. Association for Computational Linguistics.
  5. Large language models are reasoning teachers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pages 14852–14882. Association for Computational Linguistics.
  6. Cong Jiang and Xiaolei Yang. 2023. Legal syllogism prompting: Teaching large language models for legal judgment prediction. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, ICAIL 2023, Braga, Portugal, June 19-23, 2023, pages 417–421. ACM.
  7. Is chatgpt A good translator? A preliminary study. CoRR, abs/2301.08745.
  8. A general approach for predicting the behavior of the supreme court of the united states. PloS one, 12(4):e0174698.
  9. ML-LJP: multi-law aware legal judgment prediction. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023, pages 1023–1034. ACM.
  10. Learning to predict charges for criminal cases with legal basis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2727–2736. Association for Computational Linguistics.
  11. Multi-defendant legal judgment prediction via hierarchical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 2198–2209. Association for Computational Linguistics.
  12. Orca: Progressive learning from complex explanation traces of GPT-4. CoRR, abs/2306.02707.
  13. Stuart S Nagel. 1963. Applying correlation analysis to case prediction. Tex. L. Rev., 42:1006.
  14. Swiss-judgment-prediction: A multilingual legal judgment prediction benchmark. In Proceedings of the Natural Legal Language Processing Workshop 2021, NLLP@EMNLP 2021, Punta Cana, Dominican Republic, November 10, 2021, pages 19–35. Association for Computational Linguistics.
  15. OpenAI. 2023. GPT-4 technical report. CoRR, abs/2303.08774.
  16. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 5687–5711. Association for Computational Linguistics.
  17. Rechtstheorie mit juristischer Methodenlehre. Beck.
  18. Jeffrey A Segal. 1984. Predicting supreme court cases probabilistically: The search and seizure cases, 1962-1981. American Political Science Review, 78(4):891–900.
  19. A comprehensive evaluation of large language models on legal judgment prediction. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 7337–7348. Association for Computational Linguistics.
  20. Exploring the use of text classification in the legal domain. In Proceedings of the Second Workshop on Automated Semantic Analysis of Information in Legal Texts co-located with the 16th International Conference on Artificial Intelligence and Law (ICAIL 2017), London, UK, June 16, 2017, volume 2143 of CEUR Workshop Proceedings. CEUR-WS.org.
  21. Llms - the good, the bad or the indispensable?: A use case on legal statute prediction and legal judgment prediction on indian court cases. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 12451–12474. Association for Computational Linguistics.
  22. A survey on large language model based autonomous agents. Frontiers Comput. Sci., 18(6):186345.
  23. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022.
  24. Precedent-enhanced legal judgment prediction with LLM and domain-model collaboration. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 12060–12075. Association for Computational Linguistics.
  25. Lawformer: A pre-trained language model for chinese legal long documents. AI Open, 2:79–84.
  26. CAIL2018: A large-scale legal dataset for judgment prediction. CoRR, abs/1807.02478.
  27. C-pack: Packaged resources to advance general chinese embedding. CoRR, abs/2309.07597.
  28. Distinguish confusing law articles for legal judgment prediction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 3086–3095. Association for Computational Linguistics.
  29. mt5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 483–498. Association for Computational Linguistics.
  30. Neurjudge: A circumstance-aware neural framework for legal judgment prediction. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pages 973–982. ACM.
  31. Disc-lawllm: Fine-tuning large language models for intelligent legal services.
  32. Han Zhang and Zhicheng Dou. 2023. Case retrieval for legal judgment prediction in legal artificial intelligence. In Chinese Computational Linguistics - 22nd China National Conference, CCL 2023, Harbin, China, August 3-5, 2023, Proceedings, volume 14232 of Lecture Notes in Computer Science, pages 434–448. Springer.
  33. Contrastive learning for legal judgment prediction. ACM Trans. Inf. Syst., 41(4):113:1–113:25.
  34. Aligning instruction tasks unlocks large language models as zero-shot relation extractors. In Findings of the Association for Computational Linguistics: ACL 2023, pages 794–812.
  35. Charge prediction by constitutive elements matching of crimes. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 4517–4523. ijcai.org.
  36. A survey of large language models. CoRR, abs/2303.18223.
  37. Legal judgment prediction via topological learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 3540–3549. Association for Computational Linguistics.
  38. Large language models for information retrieval: A survey. CoRR, abs/2308.07107.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Chenlong Deng (7 papers)
  2. Kelong Mao (23 papers)
  3. Yuyao Zhang (52 papers)
  4. Zhicheng Dou (113 papers)