GraphER: An Efficient Graph-Based Enrichment and Reranking Method for Retrieval-Augmented Generation

Abstract: Semantic search in retrieval-augmented generation (RAG) systems is often insufficient for complex information needs, particularly when relevant evidence is scattered across multiple sources. Prior approaches to this problem include agentic retrieval strategies, which expand the semantic search space by generating additional queries. However, these methods do not fully leverage the organizational structure of the data and instead rely on iterative exploration, which can lead to inefficient retrieval. Another class of approaches employs knowledge graphs to model non-semantic relationships through graph edges. Although effective in capturing richer proximities, such methods incur significant maintenance costs and are often incompatible with the vector stores used in most production systems. To address these limitations, we propose GraphER, a graph-based enrichment and reranking method that captures multiple forms of proximity beyond semantic similarity. GraphER independently enriches data objects during offline indexing and performs graph-based reranking over candidate objects at query time. This design does not require a knowledge graph, allowing GraphER to integrate seamlessly with standard vector stores. In addition, GraphER is retriever-agnostic and introduces negligible latency overhead. Experiments on multiple retrieval benchmarks demonstrate the effectiveness of the proposed approach.

• The paper presents a novel GraphER framework that decouples offline graph-based enrichment from online reranking, addressing limitations in semantic-only retrieval.
• It introduces innovative methods like Graph Cohesive Smoothing and GAT-based ranking to enhance perfect recall at rank K for multi-hop and table retrieval tasks.
• GraphER is retriever-agnostic, efficient with sub-second online reranking, and adaptable to various domains with structural, conceptual, and contextual proximities.

GraphER: Graph-Based Enrichment and Reranking for Retrieval-Augmented Generation

Motivation and Problem Setting

Retrieval-augmented generation (RAG) systems frequently rely on semantic search techniques that map queries and data objects into embedding spaces, ranking candidates solely based on vector proximity. However, semantic similarity is limited for information needs where relevant evidence is distributed across multiple objects connected by non-semantic relationships. Standard neural retrieval paradigms fail to leverage inter-object dependencies, resulting in incomplete retrieval—especially pronounced in scenarios such as SQL table retrieval, multi-hop QA, or document chunking—where structures like foreign keys, named entity co-occurrence, or chunk adjacency imply vital non-semantic relationships.

Conventional remedies fall into agentic retrieval strategies, alignment-oriented approaches, or knowledge graph (KG)-based augmentation. Agentic methods iteratively reformulate queries using LLMs but are computationally inefficient due to their lack of corpus structure awareness. Alignment-oriented methods integrate content/structure reasoning in decoding but with heavy online LLM usage, impairing practical deployment. KG-augmented retrieval captures organizational links at the expense of maintenance complexity and incompatibility with prevalent vector store workflows.

The GraphER Framework

GraphER (Graph-based Enrichment and Reranking) addresses these inefficiencies by decoupling enrichment and reranking: all heavy enrichment is performed offline during indexing, and graph-based reranking operates only on candidate retrievals during online query time. Crucially, it abstains from constructing a full knowledge graph, is retriever-agnostic (compatible with standard vector stores), does not invoke LLMs during online retrieval, and is efficient enough for production RAG infrastructure.

Offline Graph-Based Enrichment

The offline component independently augments each data object with proximity signals along three axes:

• Structural proximity: Encodes corpus-specific relational structures, e.g., foreign keys in relational tables or hyperlinks in web corpora, via structural connectivity metadata.
• Conceptual proximity: Adds named entity annotations extracted by instruction-tuned LLMs; overlapping entity sets across objects define edges at retrieval time.
• Contextual proximity: Documents split into chunks record unique IDs, forming edges between sequential chunks.

The enrichment pass is linear in corpus size, sidestepping expensive pairwise operations.

Online Reranking via Lightweight Graph Construction

Upon query, a base retriever proposes the top-$n$ candidates. From this candidate set, a graph is instantiated: nodes are candidate objects, and edges correspond to any qualifying proximity (structural, conceptual, contextual) flagged in enrichment. Edge weights are assigned according to task-dependent heuristics (e.g., normalized entity overlap or binary adjacency for structure/context).

Three reranking strategies are evaluated:

• Personalized PageRank (PPR): Baseline graph-based diffusion, but susceptible to “hubness”—preferentially over-ranking highly linked nodes regardless of semantic relevance.
• Graph Cohesive Smoothing (GCS): A proposed iterative smoothing scheme that uses row-normalized adjacency. It updates scores by convexly combining the base retriever’s seed scores and the neighbors’ averages, returning the elementwise maximum to prevent dilution of high-confidence nodes.
• Graph Attention Network (GAT)-based Ranking: A deep model leveraging GATv2; for each node, concatenates the GCS score, the query embedding, and the object embedding, propagating higher-order dependencies through five GATv2 layers and two FC layers, producing the final node scores.

Empirical Results

Experiments are conducted across three retrieval regimes:

• Table retrieval: (Spider 1.0, Bird, Beaver) Structural proximity is operationalized via foreign-key graph structures.
• Multi-hop QA: (HotpotQA, 2WikiMultihopQA, MuSiQue) Conceptual proximity is grounded in named entity overlap.
• Chunked document retrieval: (BEIR-NQ) Contextual proximity is realized by connecting adjacent chunks.

Metrics focus on perfect recall at rank $K$ (PR@K), which demands all relevant objects must be present in the top-$K$. This is a stringent metric, critical for downstream generation.

Key empirical findings:

• GraphER-GCS consistently improves over base semantic retrievers, with median PR@10 absolute gains of 1–20% across diverse settings.
• GraphER-GAT typically matches or exceeds GCS (13/15 PR@10 cases), demonstrating the utility of non-linear inter-node modeling.
• PPR is unreliable due to its hub bias, occasionally resulting in substantial retrieval regressions.
• Improvements are robust to embedding choice and transfer across datasets, with hyperparameters primarily tuned on Bird but successful on disparate evaluation sets—a strong indicator of generalizability.

Notably, latency evaluation shows sub-second reranking per 200 candidates (0.49–0.55 seconds) on modern server hardware, vastly eclipsed by LLM generation times.

Theoretical and Practical Implications

GraphER’s decoupled design extends conventional RAG retrieval by allowing for arbitrary, application-driven definitions of non-semantic proximity. This framework elevates the completeness and relevance of retrieval in knowledge-intensive tasks without the maintenance overheads of KGs or the runtime inefficiencies of LLM-centric strategies.

The introduction of GCS addresses a standing problem in prior graph-based reranking—mitigating hub node overemphasis in PPR—while the GAT module capitalizes on cross-object feature interactions that cannot be captured by linear smoothers, further closing the recall gap for multi-hop and compositional queries.

GraphER’s retriever/model-agnosticity and negligible inference overhead render it suitable for integration into large scale, industrial RAG pipelines. The demonstrated flexibility of proximity types also points toward immediate adaptability for domains requiring, e.g., document layout proximity, code dependency graphs, or regulatory links.

Potential Directions and Impact on Future AI Systems

Future extensions could include:

• Automated induction of custom proximity types via unsupervised LLMs.
• Extension to long-context modeling by leveraging multi-hop or hierarchical proximity propagation.
• Integration with online learning for continuous proximity adaptation as data evolves.

The plug-and-play architecture of GraphER offers a generic avenue for augmenting RAG with corpus- or domain-specific structure, setting a standard for next-generation hybrid semantic-structural retrieval in LLM-centric architectures.

Conclusion

GraphER systematically addresses incompleteness in semantic-only retrieval for RAG by introducing a post-hoc, graph-based reranking pipeline leveraging corpus-specific proximity. Results demonstrate robust, transferable improvements over state-of-the-art semantic retrievers at a fraction of the computational cost of LLM- or KG-based alternatives. The coexistence of offline enrichment, flexible definition of proximity, and online, efficient reranking positions GraphER as a practical, theoretically sound methodology for enhanced retrieval in real-world RAG deployments (2603.24925).