Use Graph When It Needs: Efficiently and Adaptively Integrating Retrieval-Augmented Generation with Graphs

Abstract: LLMs often struggle with knowledge-intensive tasks due to hallucinations and outdated parametric knowledge. While Retrieval-Augmented Generation (RAG) addresses this by integrating external corpora, its effectiveness is limited by fragmented information in unstructured domain documents. Graph-augmented RAG (GraphRAG) emerged to enhance contextual reasoning through structured knowledge graphs, yet paradoxically underperforms vanilla RAG in real-world scenarios, exhibiting significant accuracy drops and prohibitive latency despite gains on complex queries. We identify the rigid application of GraphRAG to all queries, regardless of complexity, as the root cause. To resolve this, we propose an efficient and adaptive GraphRAG framework called EA-GraphRAG that dynamically integrates RAG and GraphRAG paradigms through syntax-aware complexity analysis. Our approach introduces: (i) a syntactic feature constructor that parses each query and extracts a set of structural features; (ii) a lightweight complexity scorer that maps these features to a continuous complexity score; and (iii) a score-driven routing policy that selects dense RAG for low-score queries, invokes graph-based retrieval for high-score queries, and applies complexity-aware reciprocal rank fusion to handle borderline cases. Extensive experiments on a comprehensive benchmark, consisting of two single-hop and two multi-hop QA benchmarks, demonstrate that our EA-GraphRAG significantly improves accuracy, reduces latency, and achieves state-of-the-art performance in handling mixed scenarios involving both simple and complex queries.

• The paper introduces EA-GraphRAG, a tri-routing framework for adaptive query retrieval that uses syntactic features to assign retrieval power proportionally to query complexity.
• Researchers found a 13.4% increase in accuracy on Natural Questions and a 16.6% improvement on time-sensitive queries by adaptively switching between dense and graph-based retrieval methods.
• Syntactic features, including syntactic structure ratios and dependency-based structural features, have been shown to effectively predict the appropriateness of graph-based retrieval benefits

Motivation and problem statement

Graph-augmented retrieval-augmented generation (GraphRAG) systems improve reasoning depth over knowledge-intensive queries by indexing corpora as structured graphs, but they impose this machinery uniformly on all queries. The authors of "Use Graph When It Needs" (2602.03578) argue that this rigidity is the root cause of a well-documented paradox: GraphRAG outperforms vanilla RAG on multi-hop questions yet frequently underperforms it on simple factoid queries, while incurring substantially higher latency. They cite empirical evidence that GraphRAG trails vanilla RAG by 13.4% accuracy on Natural Questions (NQ) and 16.6% on time-sensitive queries. Their diagnosis is that graph-based retrieval injects noise for queries requiring only localized evidence, and that LLM-based graph construction adds token cost and update latency.

The proposed remedy, EA-GraphRAG, is a tri-routing framework: a lightweight complexity scorer assigns each query a continuous score, which routes the query to dense retrieval, graph-based retrieval, or a complexity-aware fusion of both. The central design bet is that syntactic features—rather than semantic models or LLM judges—are sufficient to predict which retrieval paradigm will succeed, at negligible per-query cost.

Framework

EA-GraphRAG has three components.

Syntactic feature constructor. Using Stanza constituency parses with Tregex pattern matching and SpaCy dependency parses, the featurizer extracts over 80 raw features spanning four categories: syntactic structure ratios in the tradition of Lu's syntactic complexity measures (e.g., clauses per T-unit, complex nominals per clause), dependency-based structural features (maximum/average dependency distance, long-range dependency counts, tree imbalance), semantic and lexical features (named entity counts and density, question type, complexity markers such as negation and passive voice), and interaction terms. Mutual-information-based selection reduces these to roughly 85 features, z-score normalized.

MLP adapter. A small MLP with feature attention and residual connections (hidden dimensions 256–128–64) maps features to a score $s(q) \in (0,1)$. Training uses a disagreement set: only queries where dense RAG and GraphRAG produced different correctness outcomes are labeled (positive if GraphRAG was correct and RAG was not, negative otherwise), with BCE-with-logits loss, label smoothing, and AdamW with cosine annealing. Queries where both paradigms agree are excluded—an assumption worth noting, since the router never learns from cases where both fail.

Score-driven routing. Two validation-tuned thresholds $\tau_L < \tau_H$ define three paths: $s(q) \le \tau_L$ triggers dense retrieval; $s(q) \ge \tau_H$ triggers graph-based retrieval; borderline queries go to weighted Reciprocal Rank Fusion (wRRF), where the graph contribution is weighted by $s(q)$ and the dense contribution by $1 - s(q)$.

The offline index follows HippoRAG2 (Gutiérrez et al., 20 Feb 2025): a heterogeneous graph whose nodes are entities/noun phrases plus corpus passages, with OpenIE relation edges, occurrence edges linking entities to source passages, and synonymy edges between semantically similar nodes. The online graph path scores query–fact similarity, reranks top facts with an LLM, derives hub-normalized entity weights as PPR seeds, runs Personalized PageRank, and reads passage scores from the diffusion vector.

Experimental results

Experiments use 1,000 sampled queries each from NQ and PopQA (single-hop) and HotpotQA and 2WikiMultihopQA (multi-hop), pooled into a 4,000-query mixed benchmark with a merged corpus of 34,239 documents. All baselines use GPT-4o-mini as generator and for graph construction, retrieving top-5 chunks. Evaluation uses contain-match accuracy and GPT-evaluated accuracy, plus Recall@k and per-query retrieval latency.

Key QA results on the mixed benchmark:

Method	Mix Acc.	Mix GPT-Acc.
GPT-4o-mini (no retrieval)	39.5	43.4
ColBERTv2	65.3	70.2
HippoRAG2	68.5	73.6
EA-GraphRAG	71.6	76.9

The largest single gain is on 2WikiMultihopQA, where EA-GraphRAG reaches 76.3 Acc. / 81.5 GPT-Acc., versus 67.7 / 72.5 for HippoRAG2 and 56.4 / 62.7 for ColBERTv2—a nearly 20-point margin over the strongest prior system. On HotpotQA the improvement over HippoRAG2 is marginal (65.9 vs. 65.5 Acc.), so the headline gains are concentrated on 2Wiki and the mixed setting rather than uniform across multi-hop datasets.

Retrieval recall shows a similar pattern: on the mixed benchmark EA-GraphRAG attains R@3 66.0 and R@5 74.9, exceeding ColBERTv2 by 8.6 and 10.0 points but HippoRAG2 by only 0.4 and 0.8 points. Notably, EA-GraphRAG does not dominate every column—it trails HippoRAG2 on HotpotQA R@3/R@5 (80.5/87.2 vs. 82.7/89.6)—indicating the router occasionally misallocates complex queries to weaker paths or that fusion dilutes graph-only performance there.

Latency results substantiate the efficiency claim: dense retrieval takes 0.08 s/query versus 3.23 s for graph retrieval (a 96.4% reduction), and the full EA-GraphRAG system averages 2.19 s/query (1.14 s without the fusion path). The full-system latency remains substantial relative to pure dense retrieval because fusion invokes both retrievers on borderline queries; the paper frames this as a tunable trade-off via $\tau_L$ and $\tau_H$ rather than a fixed operating point.

An ablation on the mixed benchmark decomposes the gains: generator-only 39.5 → +dense 65.1 → +graph 70.5 → +fusion 71.6 Acc. Fusion contributes a modest 1.1 points over simple union of both retrievers, so most of the benefit comes from having both retrieval signals available, with routing and fusion providing incremental refinement. Case studies illustrate the complementary failure modes: graph retrieval misses sentence-level date evidence for a Game of Thrones season-8 factoid, while dense retrieval fails to traverse the film→director→death-date chain for a multi-hop question.

Limitations and open questions

Several constraints temper the results. First, the adapter is trained on only 200 disagreement samples drawn from the same four benchmarks used for evaluation, raising questions about generalization to out-of-distribution query styles and domains; the paper does not report cross-dataset transfer of the router. Second, the disagreement-only training objective excludes queries where both paradigms fail, so the score $s(q)$ reflects relative paradigm preference rather than absolute answerability. Third, the syntactic-complexity hypothesis—that parse-tree features suffice to predict retrieval-path success—is validated empirically but not analyzed mechanistically; the paper does not report which selected features carry the most weight or how robust routing is under paraphrase. Fourth, the offline index still depends on LLM-based entity extraction and OpenIE, inheriting the construction-cost concerns the paper identifies in GraphRAG generally. Finally, the fusion path doubles retrieval work on borderline queries, and the thresholds' sensitivity to workload composition (the fraction of simple vs. complex queries) is not characterized.

Conclusion

EA-GraphRAG demonstrates that a lightweight, syntax-based complexity scorer can route queries between dense and graph-based retrieval with competitive accuracy and materially lower latency than always-on GraphRAG, achieving state-of-the-art mixed-benchmark results (71.6 Acc., 76.9 GPT-Acc.) and an order-of-magnitude latency reduction relative to graph-only retrieval. The evidence supports adaptive routing as a practical middle ground, though the gains over strong GraphRAG baselines are uneven across datasets and the router's training regime is narrow. The open question the paper leaves is whether syntactic complexity generalizes as a routing signal beyond balanced single-hop/multi-hop mixtures and whether the disagreement-based adapter can be trained cheaply in production settings where paradigm-correctness labels must be collected online.