Judge-Free Adjudication
- Judge-free adjudication is a family of methods that de-discretionalizes decision-making by replacing human judgment with systematic, rule-based procedures.
- It employs techniques such as random assignment, output aggregation, and deterministic execution through LLM pipelines to ensure transparency and consistency.
- These approaches are applied across legal, clinical, and technical domains, balancing efficiency with rigorous auditability and fairness safeguards.
Judge-free adjudication is a family of methods that relocates, constrains, or automates functions ordinarily associated with human adjudicators, judges, reviewers, or examiners. In the cited literature, the phrase does not denote a single institutional model. Instead, it spans random assignment procedures that remove discretionary control over who decides a case, formal operators that combine multiple outputs into one result, optimization systems that merge conflicting annotations into a gold standard, interview-based legal self-assessment, LLM-centered pipelines for clinical and legal determinations, and deterministic benchmark designs that eliminate evaluator models from scoring itself (Stern et al., 2020, Boiten, 2015, Bar-Sinai et al., 2019, Sivarajkumar et al., 21 Mar 2025, Pattnayak et al., 18 Feb 2026). This suggests that judge-free adjudication is best understood as an umbrella term for de-discretionalized decision procedures rather than as a synonym for the total disappearance of human authority.
1. Meanings and scope
The literature uses “judge-free adjudication” in several non-equivalent senses. Some works target the selection of decision-makers rather than merits decisions; some target aggregation of multiple decisions; some automate bounded, repetitive determinations; and some eliminate judges only in the evaluation loop.
| Sense | Mechanism | Representative sources |
|---|---|---|
| Procedural de-discretionalization | Random assignment, sortition, cryptographic draws | (Stern et al., 2020) |
| Output aggregation | Bag-based adjudication operators, ASP optimization, majority-plus-payments | (Boiten, 2015, Schüller, 2018, Caragiannis et al., 2022) |
| Runtime automated determination | LLM pipelines, multimodal reasoning, deterministic graph execution | (Sivarajkumar et al., 21 Mar 2025, Afane et al., 21 Apr 2026, Wu et al., 18 Mar 2026, Sójka et al., 4 May 2026) |
| Pre-adjudicative legal assessment | Interview-driven rights determination | (Bar-Sinai et al., 2019) |
| Judge-free evaluation | Deterministic parsing, rule-based scoring, significance-preserving assessment | (Otero et al., 2023, Pattnayak et al., 18 Feb 2026) |
A recurrent misconception is that judge-free adjudication must mean fully autonomous merits determination. The sources are more qualified. Randomization in legal systems is presented as a substitute for discretionary assignment of judges or jurors, not for legal reasoning on liability or guilt (Stern et al., 2020). Formal jurisprudence modeling produces a “self-assessment report rather than a legally binding determination” (Bar-Sinai et al., 2019). The cardiovascular endpoint paper explicitly frames its system as a plausible architecture for replacing a substantial portion of manual review, while also concluding that the reported performance supports deployment “most effectively” as an assistive tool rather than as a fully autonomous system (Sivarajkumar et al., 21 Mar 2025).
2. Procedural and collective forms
One major lineage treats adjudication as the removal of human discretion from procedural allocation. In legal randomization, the central claim is that randomization procedures are used “aiming to shield important decisions from spurious influences.” The core legal application is random assignment of cases to judges or courts, supplemented by juror selection and more general sortition in legal systems. The paper is emphatic that such systems must satisfy statistical honesty, cryptographic security, transparency, and auditability, with commitment-and-reveal protocols and stakeholder-contributed randomness used to prevent rerandomization and covert influence (Stern et al., 2020). Here, judge-free adjudication means discretion-free assignment, not lottery-based merits resolution.
A second lineage treats adjudication as the combination of multiple outputs into one result. In “Diversity and Adjudication,” adjudication is formalized over bags (multisets) of outputs rather than ordered tuples, with operators satisfying properties such as unanimity, majority, and permutation invariance. The paper defines adjudication operators as relations or functions , $BAG \fun VALUE$, or $BAG \pfun VALUE$, and studies majority voting, first-past-the-post, greatest lower bound, median, average, probabilistic choice, and amplification (Boiten, 2015). The general lesson is that no single universally satisfactory total deterministic operator exists across all semantic domains. Some settings require conservative ordered aggregation; others require probabilistic or partial outputs.
A third lineage replaces adjudicators with optimization over multiple imperfect judgments. The coreference paper presents “the first automatic approach for merging coreference annotations obtained from multiple annotators into a single gold standard,” subject to linguistic hard constraints and optimization criteria that minimize divergence from annotators. It represents chains as equivalence relations over mentions, uses Answer Set Programming, and supports both fully automatic and semi-automatic modes (Schüller, 2018). In a different institutional setting, decentralized adjudication in Web3 is modeled as majority voting plus payments to strategic jurors who do not intrinsically care about the merits. Under assumptions about effort and signal quality, appropriate payment functions can recover the correct adjudication outcome with high probability, but the framework also proves that a no-effort equilibrium always exists and that every good equilibrium has a bad mirror equilibrium (Caragiannis et al., 2022). This makes explicit a central tradeoff in judge-free systems: removing trusted judges does not remove equilibrium selection problems.
3. LLM-centered adjudication in applied domains
The most direct contemporary use of the term appears in automated endpoint or liability determination. In cardiovascular clinical trials, the adjudication target implemented in practice is specifically cardiovascular death versus non-cardiovascular death. The proposed pipeline has two stages. First, an LLM-based few-shot extraction system performs sentence segmentation, tokenization, entity detection, and relation detection, formalized as , and produces for each event mention a standardized event name, associated sentence(s), negation status, and event date. Second, an LLM-based Tree of Thoughts adjudicator reasons over extracted evidence and CEC-style guidelines, formalized as , branching through acute myocardial infarction, sudden cardiac death, heart failure, stroke, cardiovascular procedure, cardiovascular hemorrhage, other cardiovascular causes, non-cardiovascular causes, and undetermined (Sivarajkumar et al., 21 Mar 2025). The system achieved extraction precision $0.96$, recall $0.71$, F1-score $0.82$, negation detection accuracy $0.86$, date extraction accuracy $0.81$, and adjudication accuracy $BAG \fun VALUE$0; CLEART evaluation on 100 patients produced an overall score of $BAG \fun VALUE$1, with particularly low timeline accuracy $BAG \fun VALUE$2. The paper explicitly concludes that this supports judge-assistive deployment rather than unsupervised replacement.
A closely related problem is deciding when not to adjudicate. In unemployment insurance adjudication, the central obstacle is “factual presumptuousness”: models issue confident determinations when critical information is missing. The benchmark contains 250 questions based on Colorado UI law, with 56% inconclusive cases. Standard RAG-based systems achieve an average of only 15% accuracy on insufficient-information cases, whereas SPEC, a checklist-driven framework that requires explicit identification of missing information before any determination, achieves 89% overall accuracy and outputs inconclusive when critical gaps remain (Afane et al., 21 Apr 2026). This is a fundamental design point for judge-free adjudication: lawful automation requires a non-decision state, not merely a better classifier.
Ride-hailing liability disputes show a multimodal variant of the same problem. RideJudge formalizes the case as $BAG \fun VALUE$3 and the decision as $BAG \fun VALUE$4, where $BAG \fun VALUE$5 belongs to a hierarchical liability label space. The framework combines SynTraj for synthetic trajectory grounding, Adaptive Context Optimization for rule pruning and precedent distillation, a Chain-of-Adjudication with an Adjudicator, Visual Analyst, and Reasoning Refiner, and Ordinal-Sensitive Reinforcement Learning for severity calibration (Wu et al., 18 Mar 2026). On real-world DiDi Chuxing benchmarks, RideJudge-8B reaches 88.41% overall accuracy, outperforming 32B-scale baselines, and the ablations show large gains from context refinement and ordinal reward shaping. The paper’s contribution is not simply higher classification accuracy; it is the attempt to make quasi-judicial platform review evidence-grounded, rule-grounded, and procedurally explicit.
4. Formalized and deterministic execution
A distinct research path avoids runtime probabilistic adjudication altogether by compiling legal or policy text into executable structures. In the PolicyModels framework, jurisprudence is represented as a policy space, a decision graph, automatic inferrers definitions, and textual localizations. The policy space is a discrete multidimensional space of legally relevant states; the decision graph contains typed nodes such as [ask], [set], and [call]; and updates are monotone, since “for each dimension, these updates can only move the location upwards” (Bar-Sinai et al., 2019). The end-of-employment model contains 74 dimensions and was developed with Kav LaOved to help workers “self-assess their legal rights.” Its outputs include legal status, rights, obligations, restrictions, and recommendations, but it does not issue binding rulings. This is judge-free adjudication only in the bounded sense of quasi-adjudicative rights diagnosis.
The neuro-symbolic DACL system pushes further toward autonomous runtime execution. “Amortized Intelligence” uses an LLM once to translate a legal text into Deterministic Autonomous Contract Language, a typed graph intermediate representation. Runtime adjudication then becomes deterministic graph execution: $BAG \fun VALUE$6, with the explicit claim that $BAG \fun VALUE$7 (Sójka et al., 4 May 2026). The system is evaluated against runtime LRM baselines and achieves 99.5% overall accuracy, 9.9x lower total token consumption than GPT-5.2 Medium, and compute-cost reductions described as “over 90%” in the abstract and “$BAG \fun VALUE$8” in the contribution summary. It has been deployed for over 12 months across 150+ agreements and about 1,000 monthly billing events. At the same time, the paper is explicit that human review of the generated graph is mandatory for production readiness, and that the current method does not yet support defeasible reasoning, open-textured standards such as “reasonable care,” or higher-order logic. The result is functionally judge-free at runtime for computational legal clauses, but only under design-time governance.
This suggests a structural distinction within judge-free adjudication. One family keeps semantic inference at runtime and tries to discipline it with prompts, checklists, or agent decompositions. Another family moves interpretive work upstream into a compiled artifact and replaces runtime judging with deterministic execution. The latter gains consistency and auditability, but only where the domain is formalizable.
5. Evaluation, auditability, and judge robustness
Judge-free adjudication also appears as a methodology for evaluating systems without relying on evaluator judges. In information retrieval, the document-adjudication paper argues that low-cost adjudication methods should not be assessed only by rank correlation. It defines significance-preservation sets $BAG \fun VALUE$9 and $BAG \pfun VALUE$0, precision and recall over significantly different system pairs, Active Agreements and Disagreements, Mixed Agreements and Disagreements, and publication bias. The key empirical finding is that methods with very high Kendall’s $BAG \pfun VALUE$1 do not always best preserve statistically significant differences, especially when the underlying pool is shallow; on DL21, publication bias remains roughly 39–58% at 9% budget (Otero et al., 2023). The implication is that low-judge or judge-free evaluation is scientifically acceptable only if it preserves inferential conclusions, not merely rankings.
IndicJR offers a stricter form of judge-free benchmarking by removing evaluator models entirely. It evaluates jailbreak robustness across 12 South Asian languages and 45,216 prompts using deterministic parsing, schema checks, multilingual refusal detectors, lexical leakage checks, and canary matching. In the JSON track, outputs must satisfy a refusal contract with REFUSE|COMPLY|ABSTAIN; in the FREE track, adjudication relies on rule-based multilingual refusal detection rather than an LLM judge (Pattnayak et al., 18 Feb 2026). The benchmark reveals a strong contract gap: under JSON, some models still exceed 0.92 JSR, while in FREE nearly all models reach approximately 1.0. Human audits on 600 samples report $BAG \pfun VALUE$2 unweighted and $BAG \pfun VALUE$3 weighted, with overall schema validity 95.4%. Here, judge-free adjudication means evaluator-free scoring.
By contrast, the LLM-as-a-Judge literature documents how fragile judge-centered evaluation remains. FairJudge reframes judging as a learnable policy $BAG \pfun VALUE$4, trained with SFT, DPO, and GRPO to improve adaptivity, debiasing, and pointwise-pairwise consistency, reaching 65.52% consistency on FairJudge-Benchmark-1K (Yang et al., 6 Feb 2026). Prompt optimization on LEXam shows that lenient judge feedback transfers better across judges than strict feedback, whereas strict judges induce narrow prompt overfitting (Elganayni et al., 22 Apr 2026). Yet the Polish National Appeal Chamber study shows how severe the failure mode can be: human examiners scored model-written judgments at 37, 30, and 8 out of 100, while GPT-4o as evaluator assigned 88, 90, and 85, including 28/30 for “Legal Provisions” where humans assigned all three models 0 (Karp et al., 6 Nov 2025). The broader lesson is that judge-free evaluation is often easier to justify than judge-free merits adjudication precisely because evaluator judges are themselves unstable.
6. Limits, fairness, and governance
The normative boundary of judge-free adjudication is not only technical; it is jurisprudential. “Causal Equal Protection as Algorithmic Fairness” treats adjudication itself as a classification system and argues that fairness should be defined not by simple classification parity, but by whether individuals are subjected to greater risks of false positives or false negatives because they possess protected characteristics (Bello et al., 2024). The paper distinguishes predictive from diagnostic evidence, argues that causal structure matters, and even contends that explicit use of protected characteristics may be required if it equalizes these risks. For judge-free adjudication, this means that “blind” automation is not automatically fair. A system can be formally consistent and still allocate legal error risk in a causally unequal way.
Governance-oriented work therefore tends to proceduralize adjudication rather than merely automate it. PaperJury is exemplary: it rejects the judge-centered loop in favor of a deterministic-versus-semantic split, a durable ledger, contestability-based routing, a due-process trial, risk-proportional guard chains, and terminal outcomes of invalid-drop, valid-fixable, and author-required (Wang et al., 15 Jun 2026). In its expert-review evaluation, it reports verdict agreement 0.887, routing agreement 0.913, and ESVR 0.025, outperforming an LLM-as-Judge review-revise loop. The paper’s core thesis is that “load-bearing safety and completion logic should reside in deterministic orchestration rather than model discretion.” This suggests a broader pattern: judge-free adjudication often succeeds not by eliminating semantic judgment, but by subordinating it to explicit process.
Across the literature, the strongest limitation is scope. Randomization does not decide legal merits (Stern et al., 2020). Clinical LLM adjudication still underperforms expert committees and lacks calibrated uncertainty and formal escalation rules (Sivarajkumar et al., 21 Mar 2025). DACL-style systems require reviewable compiled artifacts and are bounded to computational clauses (Sójka et al., 4 May 2026). High-stakes legal writing and legal evaluation remain well below human threshold in authentic exam settings (Karp et al., 6 Nov 2025). The cumulative record therefore supports a restrained conclusion. Judge-free adjudication is technically credible where the task is structured, repetitive, auditable, and rule-bound; where abstention or escalation is explicit; and where evaluation can be detached from evaluator-model discretion. It is much less credible as a general replacement for human judgment in open-textured, evidentially incomplete, or institutionally contested domains.
