- The paper introduces the first dedicated Sinhala legal NLP corpus, containing 2,022,207 tokens from 1,206 Acts and Bills published between 1981 and 2014.
- The authors combine Google Document AI OCR, filtration, and native-speaker manual cleaning, reducing the dataset from 2,865 PDFs to 1,206 documents while addressing seven major OCR error categories.
- The corpus shows substantial domain shift from general Sinhala (Jensen–Shannon divergence 0.614), while six language models achieve lower legal-text perplexity because repetitive legislative phrasing makes the domain more predictable.
SinhaLegal is a benchmark corpus of Sinhala legislative texts comprising approximately 2 million word tokens across 1,206 legal documents: 1,065 Acts (1981–2014) and 141 Bills (2010–2014). The corpus addresses a concrete resource gap: while legal NLP datasets exist for English, Chinese, German, Hindi, and other languages, Sinhala — a low-resource language by the categorisation of Ranathunga and De Silva despite its ~16 million L1 speakers — previously had no dedicated legal-domain dataset. The corpus was derived from the Sri Lanka Document Dataset (Senaratna, 5 Oct 2025), restricted to its Acts and Bills, and processed into a machine-readable form with per-document metadata.
Corpus construction
The construction pipeline proceeded through acquisition, organisation, OCR extraction, filtration, and manual post-processing. A raw collection of 2,865 PDFs (Acts and Bills only; Gazettes were excluded due to multi-column layouts that degrade OCR accuracy) was extracted using Google Document AI, chosen over alternatives such as Surya based on prior comparative evaluations on real scanned Sinhala documents (Jayatilleke et al., 24 Jul 2025, Jayatilleke et al., 22 Sep 2025). Documents exceeding 15 pages were chunked to accommodate the engine's per-request limit.
Filtration reduced the collection substantially. The authors retained only Acts from 1981–2014 and Bills from 2010–2014, excluded 96 Acts from 1992, 1996, and 1997 with double-sided printing causing severe OCR errors, removed documents over 50 pages (which tend to contain tables and complex layouts), and dropped documents with tables or multi-column formatting. Of the initial 2,865 documents, 1,206 survived — a retention rate of roughly 42%. This aggressive filtering means the final corpus systematically under-represents long and structurally complex legislation, a point the authors acknowledge explicitly.
Post-processing was performed manually by native Sinhala speakers and addressed seven error categories: word-level OCR corrections, footer and page-number removal, removal of stray sentences, seal content and prices, repeated titles, spacing normalisation, and removal of spurious characters. On a 100-document evaluation subset (~184,000 words), post-processing reduced average document length from 2,113 to 1,840 words, reflecting the removal of non-content material.
Corpus statistics and lexical analysis
The corpus contains 2,022,207 word tokens and 39,169 unique types, with an average of 1,677 tokens per document (median 1,213; range 95–23,430). Length-normalised type-token ratio (Herdan's C) is 0.7284 overall, slightly higher for Bills (0.7456) than Acts (0.7315). Notably, 46.14% of vocabulary types are hapax legomena (18,074 types), which the authors attribute to the coexistence of formulaic legal phrasing with a long tail of rare items such as bill titles and technical terminology.
Word frequency analysis reveals strong concentration: the top 100 words account for 45.89% of Act tokens and 46.39% of Bill tokens, dominated by conjunctions and particles. Frequent bigrams include formulaic constructions such as "shall be" variants and "this Act". This concentration has a direct consequence for LLM evaluation, discussed below.
A rule-based NER pipeline using regular expressions and keyword dictionaries extracted 28,937 entities across six categories:
| Entity type |
Count |
| Date |
13,532 |
| Title |
8,736 |
| Organisation |
4,281 |
| Law |
2,255 |
| Person |
126 |
| Amount |
7 |
The extreme skew toward dates and titles, with only 126 person and 7 amount entities, suggests the rule-based patterns for those categories are either too restrictive or that such entities are genuinely rare in legislative (as opposed to judicial) text. The paper concedes that NER evaluation was not automated and would require manual assessment, so no precision or recall figures are reported — a significant limitation for anyone intending to use these extractions as annotations.
Topic modelling with LDA (Gensim), set to ten topics after coherence analysis suggested fifteen but was reduced for interpretability, identified themes centred on acts themselves, courts, councils, commissions, elections, money, pensions, and towns. The term "Act/Law" dominates multiple topics, consistent with the formulaic structure observed in the frequency analysis.
LLM perplexity
Perplexity was computed on balanced samples of 2,000 sentences each from SinhaLegal and the general-domain MADLAD CulturaX Sinhala dataset, sampled via KMeans clustering to preserve diversity. The Jensen-Shannon divergence between the two corpora's word distributions was 0.614, quantifying substantial domain shift.
| Model |
MADLAD CulturaX |
SinhaLegal |
| Llama-3.1-8B |
3.05 |
2.55 |
| Deepseek-1.3B |
3.30 |
2.94 |
| Mistral-7B |
3.68 |
3.18 |
| Falcon-7B |
2.77 |
2.61 |
| DistilGPT-2 |
6.45 |
5.77 |
| Gemma-2B |
8.75 |
5.59 |
All six models exhibited lower perplexity on the legal corpus than on general Sinhala text. The authors attribute this counterintuitive result — given that legal vocabulary is more specialised — to the repetitive structures and high-frequency formulaic phrases documented in the coverage analysis, citing work on repetition phenomena in LLMs (Yao et al., 19 Apr 2025). This finding implies that low perplexity should not be interpreted as evidence of domain competence for legal Sinhala; models may find such text predictable without understanding it. Llama-3.1 and Falcon-7B performed best overall; Gemma-2B showed the largest gap between corpora.
Limitations and open questions
The paper states several limitations plainly. Coverage is restricted to Sinhala versions of Acts and Bills, excluding official English and Tamil versions and all post-2014 documents. Documents are provided as continuous text without segmentation into structural units (preamble, clauses, schedules). Documents exceeding 50 pages were excluded for practical reasons of manual post-processing, biasing the corpus toward shorter legislation. The NER output lacks quantitative evaluation, and the WER/CER figures (24.07% CER, 26.87% WER before normalisation; 23.44% WER after) are computed against the manually corrected text as ground truth, meaning they measure residual OCR noise rather than fidelity to the original printed documents. An open question is whether automated structural segmentation and entity annotation could be validated at scale without the manual effort that constrained this corpus's scope.
Conclusion
SinhaLegal provides the first dedicated machine-readable corpus for Sinhala legal NLP, constructed through systematic OCR-based extraction with extensive manual cleaning, and accompanied by corpus-linguistic analysis, rule-based entity extraction, topic modelling, and cross-domain perplexity evaluation. Its principal contributions are the corpus itself and the empirical demonstration that legal Sinhala is linguistically distinct from general Sinhala (JSD 0.614) yet more predictable to existing LLMs. The corpus's utility for downstream tasks such as summarisation and information extraction remains to be demonstrated empirically, and its coverage constraints — temporal, structural, and linguistic — define the immediate boundaries within which results on this benchmark should be interpreted.