Judicial SOP-based Legal Multi-Agent System

• SLMAS is a framework that formalizes legal processes by decomposing workflows into modular, role-oriented agents following explicit procedural guidelines.
• It employs a multi-agent architecture to simulate legal deliberation and consensus, integrating modules like issue identification, legal retrieval, and judgment refinement.
• The system leverages jurisdiction-specific SOPs and comprehensive legal datasets to ensure procedural compliance, auditability, and enhanced judgment accuracy.

A Judicial Standard Operating Procedure (SOP)-based Legal Multi-Agent System (SLMAS) is a class of AI-driven frameworks that formalize, automate, and evaluate judicial reasoning and legal proceedings via explicit encoding of the canonical procedural steps taken by courts. SLMAS decomposes complex legal workflows into modular, role-specialized agents (often instantiated as LLMs or reinforcement learning policies), each responsible for a distinct phase of the judicial process and conditioned by both legal knowledge bases and jurisdiction-specific SOPs. These systems aim to model, simulate, and in some contexts optimize not only the substantive legal outputs (e.g., verdicts, opinions, argument quality) but also the process integrity, procedural correctness, and explainability mandated in real-world adjudication.

1. SLMAS Architectural Principles and Modularization

SLMASs encode the judicial workflow as a directed acyclic graph of discrete, role-bound agents, each mirroring a human-judge, counsel, or adjudicator function. The canonical division of labor—systematized in both [AgentsCourt (He et al., 2024)] and [AppellateGen (Yang et al., 4 Jan 2026)]—includes:

• Issue Identification Agent: Distills the legally material questions, often by comparing factual findings across trial stages and filtering for new appellate evidence.
• Legal Retrieval Agent: Queries a jurisdiction-specific statutory or precedent corpus, typically using hybrid retrieval (dense embedding + sparse indexing) and optionally LLM-based confidence filters.
• Debate / Deliberation Simulation Module: Alternating or round-based turn-taking between adversarial roles (plaintiff/prosecution and defendant) operating under semantic or SOP-constrained protocols.
• Preliminary Decision and Refinement Agents: Generate an initial judgment or rationale, then refine it via incremental integration of knowledge, precedents, arguments, and optionally, public sentiment or web commentary.
• Adjudicator/Consensus Module (Panel Systems): Simulates panel-based consensus finding, aggregating individual agent leanings or votes with SOP-specified thresholds.

These modules are orchestrated as deterministic or semi-deterministic pipelines, with structured (“intermediate representation”) outputs from one stage fed as input to the next, reflecting the legal syllogism: facts → issues → authorities → application → decision.

2. Encoding and Enforcement of Judicial SOP

SOPs are represented as machine-actionable checklists or rule engines (typically in JSON or YAML), itemizing permissible agent actions, sequencing constraints, and content requirements for each procedural phase. Representative elements and enforcement mechanisms include:

• SOP Step IDs and Descriptions: E.g., “SOP_TP_01: Judge must cite standard of proof and at least one statute” (Devadiga et al., 4 Sep 2025).
• Enforcement Modules: Integrate lightweight validators that check outgoing agent messages—such as instructions, arguments, or votes—against the active SOP steps (using metadata and pattern matching).
• Auditability: Violations of SOP (e.g., missing citations, skipped procedural steps) are logged and either returned for correction or flagged for human review.

A structured SOP encoding allows the SLMAS to guarantee procedural fidelity, support dynamic SOP updates per legal regime (criminal, civil, appellate), and enable cross-system benchmarking of compliance rates and procedural step coverage (Devadiga et al., 4 Sep 2025).

3. Data Foundations: Datasets and Knowledge Bases

SLMAS research leverages jurisdiction-tailored benchmark datasets and comprehensive legal knowledge bases. For example:

• SimuCourt Benchmark: 420 Chinese judgments spanning criminal, civil, and administrative cases, annotated per phase (facts, statements, legal grounds, verdict). Each instance contains explicit procedural-stage subdivisions (He et al., 2024).
• Judicial Knowledge Base: Aggregates statues, regulations, journal articles, and millions of precedent decisions, indexed for hybrid retrieval (BM25 + embeddings, e.g., BGE, Qwen3-Embedding) (He et al., 2024, Yang et al., 4 Jan 2026, Devadiga et al., 4 Sep 2025).
• AppellateGen Corpus: Over 7,351 appellate judgment pairs with explicit mapping between first- and second-instance phases (Yang et al., 4 Jan 2026).
• RAG Sources (SAMVAD): Domain-specific index of Indian statutes, the Constitution, and landmark precedents, storaged in vector-DBs (e.g., ChromaDB) with passage-level embeddings (Devadiga et al., 4 Sep 2025).

These resources enable high-fidelity simulation of judicial workflows, retrieval of controlling authorities, and robust assessment of agent reasoning grounded in authentic legal texts.

4. Algorithmic Protocols and Reasoning Mechanisms

SLMASs operationalize legal process SOPs via multi-agent interaction, retrieval, and generation algorithms. Key formalizations include:

• Debate Simulation Protocols: Alternating turns, transcript aggregation, and convergence detection (when no new arguments arise or max rounds reached) (He et al., 2024).
• Retrieval Scoring: For facts vector $Q$ and candidate document embedding $v_d$, the precedent scoring is $s(d) = \cos(Q, v_d)$; top-$k$ are then passed to the agent for review or re-ranking (He et al., 2024, Yang et al., 4 Jan 2026).
• Agent Coordination (AppellateGen): Sequential agents $$\mathcal{A}_{\text{issue}}, \mathcal{A}_{\text{retr}}, \mathcal{A}_{\text{pred}}, \mathcal{A}_{\text{write}}$$, with each taking structured input and emitting an explicit intermediate representation (issue list, statute set, predicted verdict, draft judgment) (Yang et al., 4 Jan 2026).
• Consensus Algorithms: Weighted vote aggregation (e.g., $P_\text{guilty} = \sum_i c_i p_i / \sum_i c_i$) and thresholding, or utility-based voting with disagreement penalties (Devadiga et al., 4 Sep 2025).
• Rule Engine Simulation: JSON-encoded SOPs as predicates “when (action) then (effects),” enabling environment-driven procedural gating, cost/shock application, and legal rule compliance (Badhe, 3 Oct 2025).

These protocols ensure not only correct output but legally faithful, interpretable, and auditable process tracing.

5. Evaluation Methodologies and Empirical Findings

SLMAS performance is evaluated via metrics sensitive to both outcome quality and procedural integrity:

• Legal Ground F1 (micro-averaged): Strict set match of system-generated vs. reference legal provisions; e.g., (First instance) GPT-4: 13.6%, AgentCourt: 20.3% (+8.6pts) (He et al., 2024).
• Judgment Accuracy: Element-wise measures for charges, penalties (criminal); key-point matching via LLM evaluator (civil/administrative) (He et al., 2024).
• LLM-as-a-Judge Scoring: Fact and verdict consistency, legal application, logical reasoning on Likert scales (Yang et al., 4 Jan 2026).
• Procedural Metrics: Argument grounding (fraction of case keywords used), SOP compliance rate, explanation depth (#citations per justification), procedural step coverage, and cross-run verdict consistency (Devadiga et al., 4 Sep 2025).
• Composite Exploit Score and Red-Teaming: Effective win rates, cost-inflation, calendar pressure, and flag rates of procedural “exploit chains” in adversarial legal settings (Badhe, 3 Oct 2025).

Ablation studies confirm that (i) modular SOP mapping (phase→agent) and hybrid retrieval materially improve legal-ground recall and judgment accuracy, (ii) omission of core SOP modules (e.g., issue-identification) significantly degrades performance, and (iii) SLMAS pipelines close the gap to or surpass strong proprietary baselines in select settings (He et al., 2024, Yang et al., 4 Jan 2026).

6. Domain-Specific Generalizations and Applications

SLMAS design exhibits domain and jurisdictional flexibility. Key generalizations include:

• Jurisdictional SOP Modularity: SOP definitions are encoded for update per jurisdiction—Chinese, Indian, US adversarial, appellate review—by modularizing phase/task definitions, agent prompts, and rule-sets (Devadiga et al., 4 Sep 2025).
• Civil, Criminal, and Appellate Adaptation: Systems are tailored for first-instance, second-instance (appellate), and hybrid panel/judge procedures, with appropriate SOP items for evidence, argument, consensus, and appeals (He et al., 2024, Yang et al., 4 Jan 2026).
• Red-Teaming and Policy Simulation: SLMAS embedded in simulation environments (e.g., LegalSim) facilitate exploration of procedural robustness, adversarial exploits, and what-if regulatory analyses, beyond mere verdict automation (Badhe, 3 Oct 2025).
• Explainable and Auditable Legal Reasoning: Citation chains, explicit intermediate representations, and SOP compliance logs enable human-in-the-loop interventions and downstream evaluation (Devadiga et al., 4 Sep 2025).

Table: SLMAS Exemplars by Legal Setting and Core Technologies

System/Paper	Jurisdiction/Focus	SOP Implementation
AgentsCourt (He et al., 2024)	China, trial courts	Modular agents (debate, retrieval, judgment, refinement), strict procedural mapping
AppellateGen (Yang et al., 4 Jan 2026)	China, appellate	Pipeline: Analysis → Search → Predict → Write; intermediate checkpoints
SAMVAD (Devadiga et al., 4 Sep 2025)	India, bench deliberation	SOP YAML/JSON, RAG, consensus, compliance tracking
LegalSim (Badhe, 3 Oct 2025)	US (multi-regime, adversarial sim)	JSON rules engine, PPO/bandit/LLM agents, procedural gates

7. Technical Challenges and Research Directions

Key challenges and open issues include:

• Complex Appellate Reasoning: Capturing the causal dependency across trial stages and novel evidence in appeals remains difficult for current models, even with modular SLMAS pipelines (Yang et al., 4 Jan 2026).
• Mitigating Hallucinations: Explicit SOP grounding and citation enforcement reduce, but do not eliminate, legal hallucination and error propagation.
• Adaptive SOP Enforcement: Balancing strict procedural fidelity with the flexibility to handle edge-cases or novel procedural settings requires innovation in both SOP representation and agent adaptation (Devadiga et al., 4 Sep 2025).
• Evaluation Scalability: Cross-jurisdictional, high-fidelity, and expert-reviewed datasets and metrics for full-process SLMAS evaluation remain limited and are an active area of infrastructure development (He et al., 2024, Yang et al., 4 Jan 2026).

Adoption of SLMAS blueprints supports not only improved legal AI but also the systematic analysis of procedural weaknesses, comparative legal process studies, and the development of auditable, explainable decision tools for training, simulation, and legal operations.