StateCodes: U.S. Legal Corpus
- StateCodes is a large-scale, section-structured corpus encompassing 50 states’ statutes and 45 states’ regulations with over 4.8B characters total.
- It employs a hybrid retrieval approach with fixed-size chunking (1,000 tokens with 200-token overlap) to preserve legal-semantic boundaries for effective citation.
- Integrated with LaborBench, the corpus serves as an infrastructure to evaluate retrieval-augmented generation for complex, multi-jurisdiction legal comparisons.
StateCodes is a large-scale corpus of U.S. state legal texts introduced to support retrieval-augmented generation (RAG) for statutory simplification and related legal-AI tasks. In its primary formulation, it comprises all statutory code from 50 U.S. states and state regulations or administrative codes from 45 U.S. states, organized at the level of individual sections and dated to 2023. The corpus was introduced together with LaborBench, a benchmark derived from the 2023 Comparison of State Unemployment Insurance Laws (CSUIL), and is positioned as infrastructure for evaluating whether contemporary LLM-based systems can retrieve, compare, and interpret long, heterogeneous, jurisdiction-specific legal materials at realistic scale (Hariri et al., 26 Aug 2025).
1. Corpus definition and scope
StateCodes is described as a “novel and comprehensive state statute and regulatory corpus” intended to fill the absence of an easily usable, structured collection covering nearly all underlying state statutory codes and regulations. Its role in the originating study is dual: it is both a resource for downstream retrieval and a benchmark enabler for multi-jurisdiction legal research (Hariri et al., 26 Aug 2025).
The corpus contains all statutory code from 50 U.S. states and state regulations from 45 U.S. states. The paper reports corpus-level totals in two forms. In the introduction, it gives 488M words across 1.8M sections for statutes and 268M words across 2.2M sections for regulations. In the corpus summary table, it reports 3.0B characters for statutes and 1.8B characters for regulations, totaling 4.8B characters. The abstract gives the headline size as 8.7 GB (Hariri et al., 26 Aug 2025).
StateCodes is section-structured rather than a flat text dump. This section granularity is central to the study’s design, because statutory and regulatory materials are already partitioned into legally meaningful units, and retrieval is therefore performed over those units or chunks derived from them. A plausible implication is that the corpus is intended not merely as a storage artifact, but as an operational substrate for citation-sensitive legal retrieval.
2. Sources, coverage, and construction
The corpus was sourced from Justia, with the paper noting that the state regulations on Justia are themselves sourced from Fastcase. The data are dated to 2023, matching the year of the LaborBench source material, namely CSUIL 2023 (Hariri et al., 26 Aug 2025).
The construction rationale is tied to the non-standard digital presentation of state legal materials. Although state codes and regulations are available digitally from individual states, the paper emphasizes that presentation is highly non-standardized both across states and sometimes within a state. StateCodes therefore reflects a scraping, structuring, and normalization effort over heterogeneous public-facing legal repositories.
Collection was performed with a script written to gather the laws “in parallel,” motivated by the “structure and sheer size” of the materials. The paper points to the repository https://github.com/reglab/lawscraper and states that the script can be used to collect laws for any given year. It also notes that URLs for statutes and regulations were reused as citation links in downstream retrieval prompts (Hariri et al., 26 Aug 2025).
The paper acknowledges incomplete coverage: although StateCodes is described as containing 50 states’ statutes and 45 states’ regulations, the authors note as a limitation that they were unable to access and structure a small number of states. The paper does not enumerate which five states’ regulations were absent from the final structured corpus.
3. Scale, heterogeneity, and corpus statistics
The reported descriptive statistics make clear that StateCodes is not only large but structurally uneven. Statutory sections average 265 words with SD = 817, while regulatory sections average 122 words with SD = 780. The longest statutory section reaches 332K words, and the longest regulatory section reaches 609K words (Hariri et al., 26 Aug 2025).
The paper also highlights substantial cross-state variation in corpus size. At one end, Illinois has about 4,700 sections; at the other, California has nearly 160K sections. This suggests that jurisdictional scale is itself a retrieval variable, not merely a background property.
The authors stress that StateCodes is a “highly realistic benchmark” because it contains extensive context lengths and includes over 100 sections exceeding 5K tokens. They further situate its scale relative to prior legal retrieval corpora, reporting comparison figures of about 315K characters for the COLIEE Task 3 corpus, 6.6M characters for STARD, about 40K characters for ILSI, and 49M characters for the U.S. Code. On those figures, StateCodes is materially larger than those earlier benchmarks in statutory and regulatory scale (Hariri et al., 26 Aug 2025).
| Component | Reported size | Reported sections |
|---|---|---|
| Statutes | 3.0B characters / 488M words | 1.8M |
| Regulations | 1.8B characters / 268M words | 2.2M |
| Total | 4.8B characters / 8.7 GB | — |
These statistics matter methodologically because the legal texts are long, repetitive but subtly different across jurisdictions, full of cross-references and exceptions, and organized in section structures that can be mishandled by naïve chunking. The paper uses unemployment-insurance provisions as a concrete example of this difficulty: Colorado and the federal model statute are similarly worded regarding an “employer” threshold, yet Gemini misread 20 days as 20 weeks in the federal formulation (Hariri et al., 26 Aug 2025).
4. Ingestion, chunking, and retrieval design
For RAG experiments, the paper uses a hybrid semantic + fixed-size chunking strategy. Chunking occurs only within each statute section, and chunks never span two sections. Within a section, the study uses fixed-size chunks of 1,000 tokens with 200-token overlap. Tokenization is performed with the GPT-4 tokenizer, specifically tiktoken with encoding cl100k_base (Hariri et al., 26 Aug 2025).
This design reflects a compromise between legal structure and retrieval practicality. Because documents can be extremely long, chunking is necessary for retrieval, even though the paper notes that BM25 is less naturally dependent on fixed-size vector chunks. The section boundary is thus preserved as a hard legal-semantic constraint, while sub-section chunking is introduced as a computational convenience.
For the LaborBench experiments, retrieval was performed on a filtered subset of StateCodes focused on unemployment-insurance law. The authors manually identified which statute and regulation sections in each jurisdiction corresponded to UI law and then used a regular-expression filter over citation URLs to select the relevant sections. Retrieval methods included dense retrieval with OpenAI text-embedding-3-small, Google text-embedding-004, and E5-large (e5_large_v2), along with sparse retrieval via Okapi BM25. Dense retrieval used Maximum Inner Product Search (MIPS). The system retrieved the top 5 chunks and then expanded them to full sections for prompting (Hariri et al., 26 Aug 2025).
A plausible implication is that StateCodes was designed not only as a static corpus but as a legal retrieval testbed in which section boundaries, URL-level citations, and chunk-expansion policies are first-class experimental variables.
5. Relationship to LaborBench and statutory simplification
StateCodes was introduced alongside LaborBench, a question-answer benchmark constructed from the 2023 Comparison of State Unemployment Insurance Laws (CSUIL), a Department of Labor publication compiled by lawyers over about six months and spanning 86 state comparison tables across 8 chapters. The authors say they transform CSUIL into about 3,700 questions and answers, plus a boolean supplement with 1,272 additional questions (Hariri et al., 26 Aug 2025).
The relationship between the two resources is explicit. LaborBench provides the benchmark questions and ground-truth answers derived from state-comparison tables. StateCodes provides the underlying statutory and regulatory text to be retrieved, interpreted, and cited. Each LaborBench instance is designed around table context, a question derived from a table header, and a ground-truth answer from a table entry. StateCodes therefore supplies the legal substrate for evaluating whether a model can recover the provision that justifies the benchmark answer.
This pairing is central to the paper’s critique of end-to-end “code simplification” claims. One U.S. state had claimed to eliminate one third of its state code using AI, but the paper argues that systematic evaluation of accuracy, reliability, and risk was lacking. StateCodes and LaborBench are presented as an attempt to create that evaluation infrastructure (Hariri et al., 26 Aug 2025).
More broadly, the paper treats unemployment-insurance law as a particularly suitable stress test because such laws are “notoriously complex,” often modeled on FUTA while still varying in subtle but legally significant ways across jurisdictions. This suggests that StateCodes is less a domain-limited dataset than a general corpus whose initial validation happens to occur in a domain where near-duplicate statutory language and small semantic differences are especially consequential.
6. Empirical findings, limitations, and significance
The paper’s experiments use five generators—GPT-4o mini, Gemini 1.5 Flash, Claude 3.5 Haiku, Llama 3.1 70B Instruct, and DeepSeek V3—on the boolean subset of LaborBench. For inference, it uses a two-step pipeline: first, a model answers the legal question, often with reasoning and optionally citation; second, that output is converted into JSON conforming to a dynamically generated schema via a Pydantic model. The paper reports that fewer than 20 out of over 20,000 results failed to produce a boolean (Hariri et al., 26 Aug 2025).
The principal evaluation metrics are Accuracy, Precision, Recall, and F1, with standard errors estimated via nonparametric bootstrap with . The main empirical finding is that the best overall RAG configuration reaches about 0.691 F1, while the abstract and introduction summarize the strongest RAG setting as around 0.67 F1. RAG improves performance by about 0.18 F1 relative to no retrieval, and BM25 is generally worse than dense retrieval, though not dramatically. Even with RAG, however, models still misinterpret statutory or regulatory requirements more than one third of the time (Hariri et al., 26 Aug 2025).
Citation-oriented results are similarly mixed. The study reports top-5 recall = 0.93, elicited citation accuracy of 0.73, and conditional citation accuracy of 0.77 when the correct citation appears in the top 5. Chain-of-thought citation elicitation yields only modest gains over plain RAG (Hariri et al., 26 Aug 2025).
Performance varies substantially across states. RAG F1 ranges from 0.52 (Virginia) to 0.81 (Arkansas), while baseline F1 ranges from 0.316 (Louisiana) to 0.59 (Mississippi). The improvement from retrieval also varies sharply, with California gaining only about 0.05 F1 and Idaho about 0.30. The authors report a statistically significant negative correlation between population and RAG gain, , and no significant predictors among 2,970 state covariates after multiple-testing correction (Hariri et al., 26 Aug 2025).
The paper’s limitations are explicit. Coverage is incomplete for a small number of states; the Department of Labor did not release official citation annotations for LaborBench answers; evaluation is focused on unemployment-insurance law; and most experiments emphasize boolean tasks even though the benchmark also includes categorical, integer, and text data types. The authors further caution that RAG is helpful but far from sufficient, and suggest that more sophisticated hierarchical and multi-layer retrieval methods may perform better (Hariri et al., 26 Aug 2025).
Taken together, these findings position StateCodes as a realism-oriented legal corpus rather than a solved benchmark. Its significance lies in making multi-jurisdiction statutory and regulatory retrieval measurable at a scale and heterogeneity that expose current system limits. This suggests that StateCodes is likely to be most useful as infrastructure for retrieval, citation, and comparative-law research, rather than as evidence that current LLMs can already function as end-to-end statutory simplification systems.