- The paper argues that existing legal benchmarks measure an upper bound—doctrinal reasoning applied to lawyer-prepared questions—while access to justice depends on a lower bound involving fact extraction, issue framing, and procedural context.
- A small LEXam study of 100 questions found that typos and context dilution reduced model accuracy and destabilized rankings across GPT-4.1-mini, GPT-4.1-nano, and GPT-4o-mini, though the authors stress that the experiment was illustrative and did not measure abstention.
- The paper recommends evaluating legal AI with pro se-style inputs and robustness tests because omissions, buried facts, folk-legal assumptions, and surface errors can trigger hallucination, faulty abstention, and long-context failures that unrepresented litigants may be least able to detect.
The core argument: legal reasoning versus lawyering
This paper, authored by Andrew Lou and David Shin of Yale Law School and accepted to the AI4Law Workshop at ICML 2026, advances a structural critique of legal AI evaluation. The authors distinguish between two capabilities that current benchmarks conflate. Legal reasoning, as measured by benchmarks such as LegalBench (Guha et al., 2023), LEXam (Fan et al., 19 May 2025), and LegalBench-RAG (Pipitone et al., 2024), is the application of legal doctrine to already well-formed legal questions. Lawyering is the upstream cognitive labor of extracting legally relevant facts from a noisy narrative, ordering them coherently, supplying procedural posture, and pruning irrelevant detail. Every benchmark prompt the authors are aware of arrives at the model pre-processed by a legal expert; by construction, then, existing benchmarks measure an upper bound on model capability rather than the lower bound that governs deployment for unrepresented litigants.
The paper's central claim is that access to justice depends on this lower bound. Pro se litigants—people who appear without counsel—cannot be expected to translate emotionally loaded chronologies into industry-standard pleadings. Consequently, benchmark scores cannot be interpreted as evidence that improving models will improve equitable access to legal remedies. The authors further argue that the gap may widen over time: gains from improved reasoning flow disproportionately to users who can supply lawyer-mediated inputs, while the training interventions that produce those gains may actively degrade performance under pro se-like conditions.
The empirical basis: pro se submissions differ along documentable dimensions
The first premise of the argument rests on established findings about self-representation. The scale is substantial: 27% of federal civil cases from 2000–2019 involved at least one pro se party, roughly three-quarters of the twenty million annual state court civil cases involve at least one self-represented party, and 98% of tenants in New York eviction proceedings appeared without counsel in 2013.
Beyond scale, the qualitative deficiencies of pro se submissions are well documented. A Federal Judicial Center survey of district court chief judges and clerks identified recurring problems: physically or formally hard-to-read filings, omitted material information, and salient facts buried in unreadable narratives ordered chronologically rather than legally [stienstra2011assistance]. A controlled experiment using a mock online court portal found that lay drafters were less effective at reporting legally relevant details than trained lawyers drafting from identical fact patterns, with retired judges rating lay-drafted claims lower under blinded evaluation [toycronin2022tighten]. Ethnographic work on small-claims courts similarly shows that lay litigants produce emotionally compelling narratives that diverge from the arguments adjudication requires [obarr1985litigant].
The concern is not hypothetical in the AI context either. Courts have begun encountering pro se filings bearing the marks of frontier model assistance, including fabricated citations and confidently stated but incorrect propositions, prompting recent rulings on sanctions and docket management.
Mapping pro se deficiencies onto known LLM failure modes
The second premise connects these documented deficiencies to four degradation modes established in the general ML literature:
| Pro se deficiency |
Corresponding LLM sensitivity |
Key evidence |
| Buried salient facts |
Long-context position sensitivity |
Performance degrades when relevant information sits mid-context, even in long-context-trained models [liu2024lost] |
| Omitted material facts |
Faulty abstention |
Models silently supply assumed values 41.1% of the time on underspecified prompts; accuracy drops up to twenty percentage points [yang2025prompts] |
| Folk-legal premises |
Hallucination |
Hallucination rates of 58% (GPT-4) to 88% (Llama 2) on federal court questions in 2024 [dahl2024large]; hallucinated filings persist into 2026 even at prominent law firms |
| Surface-level errors |
Typographical perturbation |
Minor typos or grammatical errors cause drops of roughly 20–33% across tasks [zhu2023promptbenchrobust] |
Two findings deserve particular emphasis because they bear directly on the widening-gap thesis. Kirichenko et al. report that abstention remains "an unsolved problem" across frontier models, and—more pointedly—that reasoning fine-tuning, which produces measurable accuracy gains on reasoning benchmarks, degrades abstention by an average of 24%, including in domains the training explicitly targets (Kirichenko et al., 10 Jun 2025). Joren et al. find that proprietary frontier models hallucinate rather than abstain when given insufficient context, and become more confident as additional but still-insufficient context is added (Joren et al., 2024). Taken together, these results suggest the very training regime that raises upper-bound benchmark scores worsens behavior under exactly the input insufficiency that characterizes pro se use. If so, pro se users face increasing rates of legal hallucination even as headline benchmark numbers improve—a divergence invisible to developers because no standard benchmark measures it.
The authors also note a distributional consequence: the population producing inputs most likely to elicit model error is also least equipped to detect errors, since pro se litigants lack both the calibration capacity of a reviewing lawyer and resistance to the perceived authority of AI output that confirms their folk-legal theories.
An embedded lawyer-centric assumption
The paper argues that current benchmark methodology implicitly assumes a qualified intermediary supplies inputs and reviews outputs. This assumption is not incorrect per se—for deployments where upstream lawyering exists, benchmark performance may be a reasonable proxy. But it becomes problematic when legal AI development is justified by access to justice goals while evaluation encodes a lawyer-centric deployment vision. Continued optimization against such benchmarks bakes the configuration in path-dependently, potentially hard to reverse once a generation of models has been selected against these targets. The authors observe that this mirrors a broader ML research pattern of optimizing on clean inputs rather than deployment-case inputs, but argue it is unusually consequential in law because the benchmark-deployment divergence maps onto an access-to-justice spectrum.
Experimental primer on LEXam
To make the argument concrete, the authors run a small perturbation study on 100 randomly sampled English multiple-choice questions from LEXam, each with four answer options, evaluated on three OpenAI models spanning size classes: GPT-4.1-mini, GPT-4.1-nano, and GPT-4o-mini. Original LEXam task instructions were preserved untouched.
Typo perturbation: one-character deletions, one-character swaps, and keyboard-adjacent swaps inserted at three densities (one typo every 2, 3, or 4 words). All models showed general accuracy degradation, with effects large enough to introduce rank instability—the relative ordering of the three models shifted between clean and distorted conditions. Notably, degradation was not strictly monotonic in typo density for all models, suggesting model-specific differences in how degraded inputs affect token allocation. GPT-4.1-nano explicitly flagged typos ("Likely a typo...") despite instructions to answer only with a choice, indicating models attend to surface noise rather than silently absorbing it.
Context dilution: two designs. In the padding-sandwich setting, the question was embedded verbatim between blocks of semantically inert scenic filler sentences (10 or 30 per block), with all newlines removed to eliminate formatting cues. In the interleaved setting, two filler sentences were inserted between each question sentence and answer choice, forcing repeated identification and discarding of distractor content. Both settings produced degradation trends consistent with the typo experiments, and again the model ranking shifted from clean to distorted inputs—even within a single vendor's model family, robustness varied and sometimes moved in opposite directions under identical perturbations.
The authors are explicit about the limits of this experiment: the sample is small, only affirmatively correct answers were scored (abstention behavior was not measured), conclusions are drawn within one model family, and the design deliberately uses a best-case distractor. Because Liu et al. found that completely unrelated filler yields better performance than partially on-topic padding [liu2024lost], the observed degradation is a conservative estimate; pro se prompts filled with plausible-but-immaterial facts would likely degrade performance more severely. The authors frame the exercise as a primer demonstrating how lower-bound robustness analysis could be incorporated into legal benchmarks, not as definitive evidence.
Limitations and open questions
The paper is candid that its contribution is argumentative and illustrative rather than conclusive. Three questions remain open. First, whether the gap exists in the legal domain at the magnitude the general ML literature predicts has not been measured—no controlled study of pro se-styled degradation across multiple legal domains yet exists. Second, which models minimize the gap is unknown; the authors explicitly caution against assuming lower-bound performance tracks upper-bound performance across models, and their own small-sample results show rank instability consistent with this caution. Third, the relative contributions of the candidate mechanisms (long-context dilution, omission/abstention failure, hallucination, surface perturbation) in the legal domain have not been decomposed, though knowing the dominant mechanism matters for remediation. Additional constraints include the single-vendor scope of the experiment, the small question sample, and the exclusion of abstention scoring—all acknowledged by the authors as appropriate targets for teams with greater compute resources.
Conclusion
The paper's contribution is a reframing: current legal benchmarks measure legal reasoning under expert-preprocessed conditions, which constitutes an upper bound, whereas access to justice claims depend on the unmeasured lower bound of performance under pro se-like inputs. Its experimental primer on LEXam provides preliminary evidence that even benign perturbations—typos and inert filler—degrade accuracy and destabilize model rankings, and the surrounding literature suggests the training interventions driving benchmark gains may simultaneously worsen abstention and increase hallucination under insufficient context. The practical upshot is that access-to-justice justifications for legal AI are currently unfalsifiable under existing evaluation infrastructure; adopting lower-bound robustness measurement, using methods already established in the broader ML literature, would make them empirically testable and could realign developer incentives toward realistic deployment conditions.