CoCR-RAG: Enhancing Retrieval-Augmented Generation in Web Q&A via Concept-oriented Context Reconstruction

Abstract: Retrieval-augmented generation (RAG) has shown promising results in enhancing Q&A by incorporating information from the web and other external sources. However, the supporting documents retrieved from the heterogeneous web often originate from multiple sources with diverse writing styles, varying formats, and inconsistent granularity. Fusing such multi-source documents into a coherent and knowledge-intensive context remains a significant challenge, as the presence of irrelevant and redundant information can compromise the factual consistency of the inferred answers. This paper proposes the Concept-oriented Context Reconstruction RAG (CoCR-RAG), a framework that addresses the multi-source information fusion problem in RAG through linguistically grounded concept-level integration. Specifically, we introduce a concept distillation algorithm that extracts essential concepts from Abstract Meaning Representation (AMR), a stable semantic representation that structures the meaning of texts as logical graphs. The distilled concepts from multiple retrieved documents are then fused and reconstructed into a unified, information-intensive context by LLMs, which supplement only the necessary sentence elements to highlight the core knowledge. Experiments on the PopQA and EntityQuestions datasets demonstrate that CoCR-RAG significantly outperforms existing context-reconstruction methods across these Web Q&A benchmarks. Furthermore, CoCR-RAG shows robustness across various backbone LLMs, establishing itself as a flexible, plug-and-play component adaptable to different RAG frameworks.

• The paper introduces CoCR-RAG, which uses AMR-based concept distillation and LLM reconstruction to fuse heterogeneous retrieved documents into concise, fact-focused contexts.
• CoCR-RAG achieves the highest AUC across all 12 tested backbone LLMs on PopQA and EntityQuestions, with gains of up to 79.03 points over vanilla RAG.
• The results show that reconstruction is most useful for noisy contexts with larger retrieval sets, although it can slightly reduce accuracy when a single retrieved document already provides sufficient evidence.

Motivation and problem statement

Retrieval-augmented generation (RAG) mitigates the parametric-memory limitations of LLMs by injecting retrieved supporting documents into the prompt, but the retrieved documents in Web Q&A settings are heterogeneous: they originate from multiple sources with divergent writing styles, formats, and granularity, and they typically contain substantial irrelevant or redundant content surrounding a small number of answer-bearing facts. The paper frames this as a multi-source information fusion problem: naive concatenation of retrieved documents can degrade factual consistency and induce hallucination, since distractor content competes with the answer evidence during inference. The authors argue that existing context-optimization methods—sentence selection, query-focused summarization, token-level pruning—operate either at a purely lexical level (which yields compact but hard-to-interpret discrete representations) or at a purely syntactic level (which preserves readability but retains noise), and that neither provides an explicitly semantic-level fusion mechanism.

The CoCR-RAG framework

CoCR-RAG is a three-component pipeline: (a) information retrieval, (b) concept-oriented context reconstruction, and (c) reconstruction-based inference. For a question $\mathcal{Q}$, the retriever (Contriever for PopQA, BM25 for EntityQuestions) fetches the top-$K$ supporting documents. The reconstruction component then replaces the raw documents with a fused, concept-oriented context before prompting the backbone LLM with a standardized template that labels the context as "facts," encouraging the model to privilege non-parametric evidence over its parametric memory. The full pipeline is factored as $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$, making the framework a drop-in replacement for the raw-context conditioning step.

AMR-based concept distillation

The core algorithmic contribution distills concepts from each retrieved document using Abstract Meaning Representation (AMR), parsed by an mBart-based multilingual parser trained on AMR 3.0. The algorithm segments the AMR graph into sentence-level subgraphs, discards edges (relations) to avoid introducing semantic content not explicitly present in the source text, and traverses the concept-carrying nodes with a depth-first search to preserve the relative adjacency of concepts. Special handling is applied to three AMR roles: :name nodes are reassembled from their dispersed :op predicates via a stack structure to keep multi-word entities intact; :wiki nodes are canonicalized against Wikipedia definitions, with :wiki taking precedence over :name when they disagree, to normalize surface variation; and date-entity nodes are consolidated into textual month representations. Pre-defined AMR ontology nodes (e.g., government-organization) are filtered out because they would inject redundant noise into the downstream LLM reconstruction, and high-frequency concepts are removed to maximize information density. A backtrace step re-aligns the abstracted AMR concepts (which drop tense and other contextual variation) with the original lexical items in the documents, ensuring the distilled concepts remain consistent with the source semantics.

Concept-oriented context reconstruction

The distilled concept set $\mathcal{C}$ is then serialized back into natural language by an LLM—LLaMA-2-13b-chat-hf in the experiments—via an Alpaca-style prompt instructing the model to "make short sentences containing all the following keywords by adding the necessary sentence elements only." This deliberately delegates relation recovery to the backbone's pre-trained linguistic competence rather than to the AMR edges, which the authors argue avoids propagating parser noise. The procedure is model-agnostic and hyperparameter-free, in contrast to Selective Context (which requires a reduction-ratio hyperparameter) and LLMLingua (which requires a target-length budget).

Experimental setup

Evaluation covers two Web Q&A benchmarks: PopQA and EntityQuestions, both grounded in Wikipedia. Twelve open-source backbone LLMs are tested, spanning GPT-Neo (1.3B/2.7B), GPT-J-6B, OPT (1.3B–6.7B), BLOOM (560M–3B), and LLaMA-2-chat (7B/13B); commercial LLMs are excluded because their frequent updates risk parameterizing the test knowledge. Baselines comprise the Vanilla raw-context setting, instruction-guided keyword extraction and summarization, Selective Context (reduction ratio 0.5), and LLMLingua (target length 200). Metrics are substring-match accuracy ($Acc$) and a trapezoidal $AUC$ over $Acc$ as a function of $K \in [1,8]$, with $K \in \{9,10\}$ excluded from $AUC$ due to small sample sizes—a concession worth noting, since the largest-context regime is only partially evaluated.

Results

The headline result is that CoCR-RAG achieves the highest $K$0 on both datasets for all twelve backbone LLMs, with positive $K$1 over Vanilla in every case. Representative $K$2 values:

Method	PopQA (LLaMA-2-13b)	PopQA (bloom-3b)	EntityQuestions (LLaMA-2-13b)	EntityQuestions (GPT-Neo-1.3B)
Vanilla	554.59	565.54	488.31	419.76
LLMLingua	585.25	551.21	523.09	432.41
CoCR-RAG	595.84	582.81	538.31	470.74
$K$3 vs. Vanilla	+41.25	+17.27	+50.00	+50.98

The largest gains occur for the LLaMA-2-chat models (+79.03 on PopQA for the 7B model), supporting a notable claim: LLMs with longer context windows are more susceptible to noise from irrelevant retrieved content, so context reconstruction yields returns comparable in importance to expanding the context window itself. CoCR-RAG also attains the lowest standard deviation across backbones (26 on PopQA, 32 on EntityQuestions), which the authors interpret as robustness of the linguistic grounding relative to baselines whose $K$4 ranges up to 80–93.

The per-$K$5 accuracy curves reveal a more nuanced picture. Gains are minimal or slightly negative at $K$6, where context entropy is already near-optimal and any reconstruction operation introduces noise; several backbones show small negative $K$7 at $K$8 in the appendix tables (e.g., GPT-Neo-2.7B at $K$9 on PopQA, $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$0). CoCR-RAG dominates most consistently at $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$1, the regime with both long contexts and adequate sample sizes. This implies a conditional deployment rule the paper states explicitly: context reconstruction should first assess whether the raw context's entropy suffices to support inference before applying any transformation.

The baseline analysis also supports the paper's two-regime taxonomy: lexical-focused methods (Keywords, SelCon) are competitive only with larger inference-capable backbones, whereas syntactic-focused methods (Summary, LLMLingua) are more robust on smaller LLMs. CoCR-RAG is positioned as combining both: lexical-granularity concept distillation for filtering, plus sentence-level reconstruction for interpretability.

Limitations and open questions

The paper is explicit about several constraints. The context-reconstruction step relies on instruction-guided LLM inference and is therefore "affected by parametric uncertainty"—the reconstruction quality depends on the reconstruction LLM itself, and no analysis is provided of reconstruction failures propagating to answer errors. The evaluation assumes all supporting documents contain the correct answer, and the prompt design deliberately forces reliance on the context; behavior under contradictory or answer-absent retrieval is untested. Retrieval quality is treated as fixed (Contriever/BM25 off-the-shelf), so interaction between retriever errors and reconstruction is unexplored. The $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$2 regime is omitted from the $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$3 computation due to sample sparsity, leaving the very-long-context behavior only partially characterized. Finally, the framework is evaluated on entity-centric factoid QA only; whether AMR-based distillation generalizes to multi-hop or abstractive QA remains an open question, as does the relative merit of alternative linguistic structures such as dependency parsing or semantic role labeling, which the authors identify as candidate replacements for AMR.

Conclusion

CoCR-RAG addresses multi-source information fusion in RAG-based Web Q&A by distilling AMR-grounded concepts from retrieved documents and reconstructing them into compact, concept-oriented contexts via an LLM. Across twelve backbone LLMs and two benchmarks, it uniformly outperforms raw-context conditioning and established compression baselines, with the largest improvements on long-context backbones and in mid-to-high $$P(\mathcal{A}|\mathcal{Q}) = P(\mathcal{A}|\mathcal{R},\mathcal{Q})P(\mathcal{R}|\mathcal{C},\mathcal{Q})P(\mathcal{C}|\mathcal{D},\mathcal{Q})P(\mathcal{D}|\mathcal{Q})$$4 regimes, and it does so without hyperparameter tuning. The evidence supports linguistically grounded semantic features as a viable basis for context-level fusion, while leaving open the robustness of the instruction-driven reconstruction step and the framework's applicability beyond entity-centric factoid QA.