- The paper presents LEGALWORLD, a novel stage-connected simulation that models the full civil litigation life-cycle for robust legal AI agent evaluation.
- It employs a dual-layer memory architecture and modular skill/tool libraries to ensure accurate cross-stage state propagation and procedural fidelity.
- Results highlight significant capability divergences among AI models, underlining courtroom advocacy as the most challenging legal reasoning task.
LEGALWORLD: Life-Cycle Simulation of Civil Litigation for Legal AI Agent Evaluation
Motivation and Context
The simulation of legal workflows for AI agents has predominantly focused on isolated subtasks or short scenario fragments, failing to capture the procedural continuity and causal dependencies inherent in real litigation. LEGALWORLD directly addresses this gap by constructing a comprehensive, stage-connected environment for Chinese civil disputes, enabling robust evaluation of procedural capabilities, cross-stage state propagation, and authentic role-specific interaction. The system is grounded in a large-scale dataset (75,309 paired first- and second-instance judgments), and supports simulation of clients, lawyers, and judges via role-bound interfaces, skill/tool libraries, and structured memory components.
Life-Cycle Environment Architecture
LEGALWORLD models civil litigation as a five-stage causal chain (consultation, complaint/defense drafting, first-instance trial, appeal drafting/response, second-instance trial), subdivided into seven scenario types. Each case trajectory is initialized from structured seeds extracted from real judgments, ensuring procedural fidelity. The environment orchestrates agent transitions, role remapping, artifact generation, and cross-stage memory updates for both clients and lawyers, while judges are instantiated as stage-specific procedural controllers.
Role-bound visibility rules rigorously enforce contextual access to facts, evidence, pleadings, and procedural records. Clients are further characterized by the Legal Client Persona Framework (LCPF), parameterizing legal literacy, disclosure willingness, emotional stability, and narrative proficiency at three levels, and validated for behavioral impact in scenario studies.
A central innovation is the dual-layer memory infrastructure: in-scenario local memory maintains turn-level continuity within each stage, while global case memory persistently records facts, evidence status, procedural progress, client goals, and confirmed positions across all stages. This enables agents to accumulate and access trajectory-wide information, mitigating state fragmentation. Memory is exposed via bounded revise/expand operations, with clients and lawyers maintaining parallel but role-specific views, accurately modeling professional versus lay cognition.
Agents are equipped with modular Skill/Tool libraries supporting procedural tasks, document drafting/export, evidence retrieval, statute checking, and memory updates. Skills serve as declarative capability guides per task and stage; tools are executable interfaces for artifact manipulation and legal information access. This formal separation between skill knowledge and tool execution ensures procedural compliance and restricts action scope at each interaction.
Dataset Construction
LEGALWORLD's case seeds are generated from extensive public judgment corpora, paired and filtered for procedural completeness (only disputes with both substantive first- and second-instance hearings are retained). Extraction uses LLM-based schema mapping for claims, factual findings, evidence, legal references, and procedural outcomes, with aggressive validation controls for data integrity. Three dataset splits (Full: 75,309 cases; Medium: 1,000; Light: 100) facilitate tractable simulation and benchmarking, maintaining cause-level composition for domain coverage.
Evaluation Protocol: LongJud-Bench
Building on the environment, LongJud-Bench systematically evaluates legal-agent capability across all litigation stages. Eight capabilities are scored—consultation issue spotting, document drafting (party identification, claim/fact/evidence marshalling), courtroom advocacy (position consistency, evidentiary advocacy, legal reasoning)—using rule-based field matching and LLM-as-Judge semantic scoring. Evaluators (217 legal professionals) provide 18,992 ratings on stage authenticity (procedural and process coherence) and role consistency (stance/motivation authenticity, role distinguishability).
The system achieves high scores: aggregate human ratings for stage authenticity (8.96/10) and role consistency (8.98/10), supported by tight agreement with LLM-as-Judge scores (64% within ±1 point for stage authenticity; 77% for role consistency). Most judges, clients, and lawyers sustain distinct and consistent trajectories over the full cycle.
Judicial Output Alignment
To validate procedural realism, simulated judgments are compared to their real counterparts on six structured dimensions (verdict, reasoning, legal reference, appeal action, entity, structure), with close alignment (overall F1 ≈ 8.28/10). The main deficit is in precise legal reference citation, indicating a need for improved statutory granularity in agent reasoning.
Cross-Model Capability Analysis
LongJud-Bench enables fine-grained profiling of multiple LLM backbones (Kimi-K2.5, Qwen3.5-Plus, GPT-5.2, DeepSeek-V4-Flash, GLM-4.7, Qwen3.5-Flash) across all stages and capabilities. Results expose sharp divergences: no backbone leads globally. Kimi-K2.5 excels at drafting, GPT-5.2 dominates courtroom advocacy (evidentiary advocacy and legal reasoning), and Qwen3.5-Plus constructs claims most effectively. Aggregated scores hide these performance trade-offs, demonstrating that trajectory-level evaluation is essential for detecting capability gaps and unbalanced strengths.
A persistent finding is that courtroom advocacy (multi-turn reasoning, evidence integration, on-the-fly legal argument) remains the hardest competency, even as formal sub-skills (party identification) saturate when procedural scaffolding is available. This crystallizes evidentiary advocacy and legal reasoning as critical targets for future model development and training.
Cross-Stage Causal Dependence
Experimental interventions show strong causal propagation: high-quality early-stage drafting improves downstream trial outcomes by +8–10 points, while low-quality drafts degrade them by −26–28 points. This validates procedural fidelity—errors and omissions in earlier stages persist and amplify, mirroring real legal practice and underscoring the necessity of modeling procedural continuity.
Trajectory-Level Training Signals
The environment's detailed procedural traces are repurposable as grounded data for agent improvement. Post-case trajectory reflection (RLS) yields reusable legal-practice rules, which, when injected as cause-matched skills, boost subsequent agent performance by +3.73 points overall (+4.20 on reflected cases, +2.34 in held-out cases). This demonstrates the utility of full-cycle procedural simulation not only for evaluation but also for training and skill transfer.
Implications and Future Directions
LEGALWORLD redefines legal-agent evaluation by making procedural capability a trajectory-level property, reframing benchmarking away from isolated task scores to life-cycle performance. The environment's modular infrastructure, memory, and skill/tool layers lay the foundation for high-fidelity legal-agent simulation, with broad dataset coverage and rigorous evaluation supporting research at scale.
Practically, LEGALWORLD enables rapid, realistic benchmarking of legal AI systems, identification of phase-specific weaknesses, and collection of causally coherent training signals. Theoretically, the architecture advances modeling of agentic workflows, memory-driven state propagation, and structured professional reasoning, informing agent design for other domains with complex procedural dependencies.
Limitations include scope restriction to Chinese civil procedure, simplification of exceptional events, and reliance on simulation benchmarks rather than real-world outcomes. Extensions should target criminal, administrative, enforcement, and retrial domains, procedural branching (jurisdictional objections, counterclaims, expert opinions), and human-agent collaborative validation.
Conclusion
LEGALWORLD constitutes the first procedural-fidelity, stage-connected environment for legal AI agent benchmarking, grounded in large-scale real-world judgments and equipped with reusable infrastructure for long-horizon simulation. LongJud-Bench exposes capability-level performance and cross-stage causal propagation, supporting robust agent evaluation and improvement. The trajectory-centric approach both elevates assessment standards and supplies grounded training signals, marking a pivotal step forward in legal AI agent research (2606.18728).
