2000 character limit reached
No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts
Published 24 Oct 2024 in cs.CL, cs.AI, cs.CY, and cs.LG | (2410.19184v2)
Abstract: In a context where the Brazilian judiciary system, the largest in the world, faces a crisis due to the slow processing of millions of cases, it becomes imperative to develop efficient methods for analyzing legal texts. We introduce uBERT, a hybrid model that combines Transformer and Recurrent Neural Network architectures to effectively handle long legal texts. Our approach processes the full text regardless of its length while maintaining reasonable computational overhead. Our experiments demonstrate that uBERT achieves superior performance compared to BERT+LSTM when overlapping input is used and is significantly faster than ULMFiT for processing long legal documents.
- Zeroberto: Leveraging zero-shot text classification by topic modeling. In International Conference on Computational Processing of the Portuguese Language, pages 125–136. Springer.
- Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
- CNJ (2024). Conselho nacional de justiça - cnj. Accessed: 2024-08-05.
- A survey on legal judgment prediction: Datasets, metrics, models and challenges. arXiv preprint arXiv:2204.04859v1.
- Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30.
- BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
- Debacer: a method for slicing moderated debates. In Anais do XVIII Encontro Nacional de Inteligência Artificial e Computacional, pages 667–678. Sociedade Brasileira de Computação-SBC.
- REALM: retrieval-augmented language model pre-training. In International conference on machine learning, pages 3929–3938. PMLR.
- Hasan, M. (2022). Transformers in natural language processing.
- Viettel-AI at SemEval-2023 task 6: Legal document understanding with longformer for court judgment prediction with explanation. In Ojha, A. K., Doğruöz, A. S., Da San Martino, G., Tayyar Madabushi, H., Kumar, R., and Sartori, E., editors, Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023), pages 862–868, Toronto, Canada. Association for Computational Linguistics.
- Universal language model fine-tuning for text classification. In Gurevych, I. and Miyao, Y., editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328–339, Melbourne, Australia. Association for Computational Linguistics.
- Using deep learning to predict outcomes of legal appeals better than human experts: A study with data from brazilian federal courts. PloS one, 17(7):e0272287.
- Natural language processing in the legal domain. arXiv preprint arXiv:2302.12039v1.
- A survey on text classification: From traditional to deep learning. ACM Trans. Intell. Syst. Technol., 13(2).
- The law and NLP: Bridging disciplinary disconnects. In Bouamor, H., Pino, J., and Bali, K., editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3445–3454, Singapore. Association for Computational Linguistics.
- ILDC for CJPE: Indian legal documents corpus for court judgment prediction and explanation. In Zong, C., Xia, F., Li, W., and Navigli, R., editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4046–4062, Online. Association for Computational Linguistics.
- Legal judgment prediction: If you are going to do it, do it right. In Preo\textcommabelowtiuc-Pietro, D., Goanta, C., Chalkidis, I., Barrett, L., Spanakis, G., and Aletras, N., editors, Proceedings of the Natural Legal Language Processing Workshop 2023, pages 73–84, Singapore. Association for Computational Linguistics.
- Hierarchical transformers for long document classification. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 838–844.
- BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593–4601, Florence, Italy. Association for Computational Linguistics.
- Long-length legal document classification. arXiv preprint arXiv:1912.06905v1.
- Big Bird: Transformers for longer sequences. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, volume 33, pages 17283–17297. Curran Associates, Inc.
- Sliding-bert: Striding towards conversational machine comprehension in long contex. Adv. Artif. Intell. Mach. Learn., 3:1325–1339.
- NLPStatTest: A toolkit for comparing NLP system performance. In Wong, D. and Kiela, D., editors, Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing: System Demonstrations, pages 40–46, Suzhou, China. Association for Computational Linguistics.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
