LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning

Abstract: Graph-based Retrieval-Augmented Generation (GraphRAG) advances flat document retrieval by structuring knowledge as relational graphs, enabling more coherent and effective reasoning. However, applying it to specific domains like legal reasoning faces critical challenges. (i) Legal corpora are heterogeneous, containing multi-granular knowledge from cases, articles and interpretations. A flat knowledge graph cannot adequately differentiate between factual details, applied rules, and abstract principles, limiting accurate retrieval. (ii) Reliable legal judgment demands transparent, evidence-based reasoning. Traditional RAG passes retrieved context directly to an LLM without verification, resulting in opaque, error-prone reasoning. To this end, we propose LegalGraphRAG, a framework designed for reliable legal reasoning. Our approach introduces two core components: a hierarchical legal graph that hierarchically organizes legal sources to enable retrieval at appropriate abstraction levels, and a multi-agent system for reliable legal reasoning, where a Researcher retrieves candidate evidence, an Auditor rigorously verifies its validity against source documents, and an Adjudicator synthesizes the set of verified evidence to render a final judgment. Extensive experiments show that LegalGraphRAG achieves the state-of-the-art performance, outperforming existing GraphRAG baselines in accurate and trustworthy legal analysis. Our code, datasets and implementation details are available at https://github.com/XMUDeepLIT/LegalGraphRAG.

• The paper introduces a multi-agent, hierarchical graph approach that decouples factual, interpretive, and rule-based evidence for improved legal reasoning.
• It employs Researcher, Auditor, and Adjudicator agents to retrieve, validate, and synthesize evidence, leading to accuracy gains of up to 19.1%.
• Experimental evaluations on legal benchmarks demonstrate significant advancements in traceable, interpretable legal predictions over traditional RAG methods.

LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning

Motivation and Limitations of Traditional RAG Methods

The domain of legal reasoning presents multi-granular and heterogeneous knowledge, encompassing factual case details, statutory rules, and abstract legal interpretations. Conventional Retrieval-Augmented Generation (RAG) approaches, which rely on flat document retrieval based purely on semantic similarity, fail to distinguish and organize these layers coherently. This limitation leads to two critical deficits: structural granularity bias—where factual details overshadow abstract principles—and lack of evidence verification, which results in error-prone, non-transparent decision-making.

Figure 1: Conventional RAG struggles with domain heterogeneity and fails to verify evidence, leading to retrieval of irrelevant information and flat graph structures.

Empirical analysis demonstrates that flat retrieval mechanisms misprioritize evidence due to surface-level overlaps, while hierarchical strategies yield 25.3% improvements in retrieval effectiveness (Figure 2). Moreover, injecting legally plausible but irrelevant content into retrieval contexts reveals significant sensitivity of traditional RAG models to context purity; performance degrades sharply—without explicit verification mechanisms, models cannot reliably distinguish valid from misleading evidence.

Figure 2: Conventional RAG retrieval is highly error-prone for heterogeneous legal documents, underscoring the necessity for structural hierarchies.

Framework Design: Hierarchical Knowledge Graph and Multi-Agent Workflow

LegalGraphRAG introduces a novel architecture combining a hierarchical legal knowledge graph (HierarGraph) with a sequential multi-agent evidence-based reasoning workflow. HierarGraph decouples three fundamental subgraphs:

• Fact Graph: Encodes precedents, linking cases, articles, and offenses to provide granular factual grounding.
• Ontology Graph: Abstracts case features, mapping narratives onto domain-specific legal properties and clustering cases via k-NN and Leiden community detection.
• Rule Graph: Links articles to judicial interpretations and equips each article with diagnostic checklists for atomic condition verification.

This structure provides both coarse and fine-grained retrieval, essential for precise reasoning across heterogeneous legal sources.

Figure 3: LegalGraphRAG architecture: Hierarchical Knowledge Construction (Fact, Ontology, Rule graphs) and sequential multi-agent reasoning (Researcher, Auditor, Adjudicator).

In the evidence-based legal reasoning phase:

• Researcher Agent: Executes parallel retrieval strategies (semantic match, community expansion, charge anchoring) to maximize evidence coverage.
• Auditor Agent: Performs rigorous validation using diagnostic checklists and interpretations, pruning inapplicable evidence.
• Adjudicator Agent: Synthesizes the validated subgraph to render interpretable, citation-supported judgments.

  Figure 4: Hierarchical Knowledge Construction phase operationalizes structured legal knowledge layers.

  

  Figure 5: Evidence-based Legal Reasoning phase: Researcher retrieves, Auditor verifies, Adjudicator synthesizes.

Experimental Evaluation and Numerical Results

Extensive evaluation on CAIL2018 and CMDL benchmarks demonstrates the superiority of LegalGraphRAG. It consistently outperforms open-source, advanced, and legal-specific baselines across all metrics, delivering accuracy gains from 6.3% up to 19.1%. Notably, LegalGraphRAG is model-agnostic, integrating flexibly with backbone LLMs and achieving peak performance (up to 78.7% on CMDL).

LegalGraphRAG also exhibits robust performance in exact law article prediction (47.9% accuracy, surpassing both SOTA RAG and massive advanced models) and term of penalty prediction (lowest MAE, strongest precision in quantitative reasoning).

Figure 6: Comparative case study: LegalGraphRAG produces correct, transparent judgments with verifiable legal evidence chains; baselines either miss relevant statutes or remain ambiguous.

LegalGraphRAG enhances retrieval effectiveness and minimizes error rates compared to flat and tree-based graph RAGs (Figure 2). Reliability analysis reveals substantial gains in traceable correctness—every prediction is explicitly backed by retrieved evidence, eliminating unsupported ‘black-box’ decisions (Figure 7).

Figure 7: Reliability analysis: LegalGraphRAG maximizes traceable correct judgments, minimizing untraceable, evidence-lacking predictions.

Figure 8: Charge prediction performance remains robust across variations in retrieval parameter $k$.

Ablation Studies and Component Analysis

The hierarchical graph structure is indispensable: removal yields a 7.2% accuracy drop. Both Researcher and Auditor agents are synergistic—exclusion degrades performance by 4.0% and 3.4% respectively, confirming the necessity of evidence coverage and rigorous verification.

Qualitative Case Studies

LegalGraphRAG achieves interpretable, evidence-grounded reasoning in real-world cases. For Dangerous Driving (Figure 9), evidence is retrieved and validated against statutory checklist conditions. For Occupational Embezzlement (Figure 10), the model reconstructs abuse of professional position using both factual and interpretative reasoning.

Figure 9: Dangerous Driving case: evidence retrieval aligns with statutory requirements.

Figure 10: Occupational Embezzlement case: model reasons through professional abuse diagnostics.

Implications and Future Directions

LegalGraphRAG demonstrates that advanced reliability and transparency in legal AI is achievable through structured multi-agent reasoning and hierarchical graph organization. This design solves fundamental issues of granularity, evidence support, and interpretability—yielding practical improvements in legal judgment prediction with real-world applicability.

Theoretical implications extend to broader domains demanding multi-granular, verifiable reasoning (medicine, finance, scientific discovery). The current unimodal textual scope of LegalGraphRAG highlights a promising future direction: the incorporation of multimodal evidence nodes (e.g., visual, audio) to enable cross-modal fact verification, further advancing ‘Smart Court’ systems and comprehensive judicial AI.

Conclusion

LegalGraphRAG establishes a new state-of-the-art paradigm for complex legal reasoning. By integrating hierarchical knowledge structuring and collaborative multi-agent verification, it advances accurate, trustworthy, and interpretable AI for high-stakes legal analysis. The evidence-based reasoning pipeline ensures each judgment is traceable to legal sources, addressing longstanding challenges in legal NLP and providing a model-agnostic foundation for future legal and domain-specific AI systems (2605.28120).