--- title: 'LEAD: Legal Agentic CoT Distillation' url: https://www.emergentmind.com/topics/legal-agentic-cot-distillation-lead type: topic --- # LEAD: Legal Agentic CoT Distillation Legal Agentic CoT Distillation (LEAD) is a supervised fine-tuning methodology specifically developed within the LegalOne project to reliably transfer rigorous, multi-step legal reasoning into large language models. LEAD augments traditional chain-of-thought (CoT) distillation with an agentic workflow that decomposes complex judicial decision-making into structured, factually grounded reasoning trajectories. By harvesting and refining reasoning fragments from authentic Chinese legal documents, then enforcing logical structure and factual alignment through a multi-stage process, LEAD addresses persistent issues in legal LLMs such as hallucinated citations, omitted intermediate inferences, and failure to mirror real-world judicial protocols [2602.00642]. ## 1. Objectives and Motivation Legal reasoning tasks are distinguished by their knowledge intensity—demanding precise authority grounding in statutes, case law, and commentary—and their structure intensity, which requires explicit, procedural, multi-hop inference (e.g., issue identification, rule application, syllogistic logic). General-purpose LLMs routinely generate legally plausible text yet struggle with domain-specific rigor; they often hallucinate sources, skip legally salient steps, or misrepresent the structure of judicial reasoning. LEAD was introduced to resolve these deficits by shifting from naive teacher-student distillation to a process that (1) harvests reasoning from authentic judicial records rather than teacher generations, (2) decomposes legal tasks into agentic sub-processes aligned with real courtroom SOPs, (3) reconstructs and merges these into highly-structured CoT trajectories, and (4) imposes a rigorous, two-phase quality control mechanism using both heuristics and an LLM-as-judge. The resulting high-consistency SFT corpus explicitly grounds legal reasoning, internalizes citations, and minimizes inherited teacher errors [2602.00642]. ## 2. Agentic Workflow and System Overview LEAD operationalizes legal reasoning as a network of “Agentic Processes,” each simulating a distinct judicial sub-procedure. The agentic pipeline comprises four sequential phases: 1. **Prompt Collection:** Extraction of diverse query forms from over 100,000 recent Chinese judicial decisions, including segmentation into Fact, Reasoning, and Decision components, citation linkage, and multi-perspective simulation of queries. 2. **Agentic CoT Synthesis:** A strong LLM (e.g., Qwen3-235B-A22B-Thinking) is tasked with executing, in sequence, sub-tasks such as Fact Finding, Issue Identification, Rule Retrieval, Rule Deduction, and Conclusion Derivation. Each node exposes the LLM to expert protocols, logical templates (“Major premise → Minor premise → Conclusion”), and legal KB retrieval as scaffolding, ensuring factual grounding and conformity with legal standards. 3. **Trajectory Refinement:** Sub-traces from each node undergo LLM-driven internalization (rewriting external references as model-held knowledge) and merging into a globally coherent reasoning trajectory. 4. **Quality Control:** Dual-stage filtering removes malformed or inconsistent samples and uses an LLM as judge to score outputs on dimensions such as reasoning quality and answer-reasoning alignment. Through this workflow, LEAD produces prompt–CoT–answer triples that directly reflect the workflow of legal professionals, promoting robust internalization of legal knowledge. ## 3. Detailed Methodological Pipeline ### 3.1 Prompt Collection - **Corpus Construction:** >100,000 court decisions are parsed and segmented using regex heuristics. Each is rated for factual completeness/complexity by a strong LLM and subjected to stratified sampling to balance case types. Statutory citations are linked to external knowledge bases. - **Question Generation:** Prompts are generated by mapping document structure (e.g., Fact→Reasoning, Fact→Decision) and simulating queries from multiple legal perspectives (litigant, attorney, judge). Real consultation queries (~20k) are appended to improve alignment. ### 3.2 Agentic CoT Synthesis A workflow graph $G=(V,E)$ links the main sub-processes: Fact Finding, Issue Identification, Rule Retrieval, Rule Deduction, Conclusions. For each prompt, the LLM traverses the graph: - Loads sub-task scaffolding; - Generates the sub-trace $τ_v$; - Retrieves relevant statutes (if required) via legal KB queries; - Iteratively appends each result to context. ### 3.3 Trajectory Refinement - **Knowledge Internalization:** LLM rewrites $τ_v$ statements to eliminate external citations, transforming them into assertions of internal knowledge while preserving logical intent. - **Reasoning Convergence:** The individual sub-traces are consolidated into a globally consistent CoT $σ$, preserving inferential steps and enforcing logical order. ### 3.4 Quality Control - **Stage I—Heuristic Filtering:** Removes truncated, malformed, or deduplicated examples; applies code-mix ratio thresholds. - **Stage II—LLM-as-Judge:** Each example is scored (1–10) for reasoning quality, consistency, answer-reasoning alignment, conciseness, fluency, and pedagogical value; examples with ratings below 7 in any category are discarded. After quality control, the resulting ~500K SFT examples are merged with ~50K high-quality open-source instructions, forming the final training dataset for supervised fine-tuning. ## 4. Training Objective and Implementation The final SFT dataset consists of triples $(q_i, σ_i, a_i)$, with chain-of-thought $σ_i = (y_{i,1}, ..., y_{i,T_i})$ and answer $a_i$. The model $p_\theta$ is optimized with the following loss: \[ \mathcal{L}_{\rm SFT}(\theta) = -\sum_{i=1}^N \sum_{t=1}^{T_i} \log p_\theta\bigl(y_{i,t}\,\bigl|\,q_i,\,y_{i,