ToE: A Hierarchical and Explainable Claim Verification Framework with Dynamic Multi-source Evidence Retrieval and Aggregation

Abstract: The rapid spread of fake news poses increasing threats to information ecosystems, especially as AI-generated misinformation under Generative Engine Optimization (GEO) poisoning allows adversarially crafted content to be systematically surfaced by retrieval systems, contaminating LLM reasoning. In this paper, we propose Tree of Evidence (ToE), a hierarchical evidence reasoning framework for automated fact-checking that models each claim as a dynamically expanding argument tree. ToE integrates a reinforcement learning-driven multi-source retrieval agent, an evidence evaluation agent, and an argument tree aggregation algorithm to iteratively decompose, retrieve, and verify claims through an explainable evidence chain. We further provide a theoretical analysis of the retrieval process, deriving a formal error bound that guarantees the learned policy converges to a neighborhood of the information-theoretically optimal policy. Experiments across multiple datasets and backbone LLMs demonstrate that ToE achieves improvements ranging from 4 to 24 percentage points over competitive baselines, with particularly pronounced gains on adversarially poisoned inputs.

• The paper introduces the ToE framework, a hierarchical system that uses dynamic multi-source retrieval and reinforcement learning to verify claims.
• It employs an evidence evaluation agent with stance-aware neural networks and neural tree aggregation to ensure transparent and traceable fact-checking.
• Empirical results on benchmarks and adversarial datasets demonstrate up to a 24 percentage point improvement over baselines, confirming its robustness.

Hierarchical and Explainable Automated Fact-Checking via Dynamic Multi-Source Retrieval: The ToE Framework

Motivation and Problem Definition

The increasing sophistication of AI-generated misinformation, particularly under Generative Engine Optimization (GEO), exposes critical vulnerabilities in Retrieval-Augmented Generation (RAG) pipelines. Malicious actors can systematically poison retrieval corpora with adversarial content, causing LLMs to surface and reason over false evidence, yielding confidently incorrect outputs and undermining trust in automated AI systems.

Figure 1: Illustration of LLM context pollution via malicious retrieval.

Traditional fake news detection methods based on stylistic cues or static LLM analysis are fundamentally limited against these attacks due to the linguistic indistinguishability of fabricated facts and their frequent exploitation of retrieval algorithms. Addressing this issue requires dynamic, targeted evidence collection and a traceable, interpretable reasoning process capable of transcending pre-trained parametric knowledge and providing verifiable verdicts.

ToE Framework Overview

The Tree of Evidence (ToE) system presents a hierarchical and explainable approach for automated claim verification, instantiated as a dynamically growing argument tree. Each node in the tree represents a (sub-)claim, which is subjected to iterative multi-source evidence collection, stance-aware evaluation, and aggregation. The framework encompasses three main components:

1. Reinforcement Learning-Based Multi-Source Retrieval Agent: Trained to allocate search actions across a heterogeneous tool set—including Wikipedia, arXiv, fact-checking databases, social media, and general web sources—disambiguating queries and initiating targeted evidence discovery until a stopping criterion is met.
2. Evidence Evaluation Agent: Employs a stance-aware neural network to compute veracity and reliability scores conditioned on the full evidence set associated with each node, factoring in evidence stance (support/refute/neutral), source credibility, provenance, and intra-group consistency.
3. Tree Management and Neural Aggregation: Orchestrates claim decomposition into sub-claims when reliability is insufficient, propagates scores bottom-up for node-level and global aggregation, and prunes verified or redundant subtrees to enforce computational efficiency.

   Figure 2: An overview of ToE framework.

Each claim is thus verified via an interpretable evidence chain, with the complete argument tree providing stepwise documentation and post-hoc analyzability.

Theoretical Foundations and Guarantees

ToE formalizes evidence collection as a Partially Observable Markov Decision Process (POMDP), in which the latent claim veracity is indirectly inferred by sequential evidence acquisition. The reward function for the retrieval agent is precisely aligned with the incremental reliability gain, provably approximating the expected mutual information between the claim label and new evidence, subject to an explicit error bound.

Under a submodular reliability function, the greedy search policy is guaranteed to achieve at least $(1-1/e)$ of the information-theoretic optimum. The retrieval reward signal's fidelity is further bounded: for reliability approximation error $\epsilon$, the per-step and cumulative policy value gaps are bounded by $2\epsilon$ and $4T\epsilon$ respectively, under a learning horizon $T$. These properties substantiate the rationality and near-optimality of RL-driven evidence search within ToE.

Core Components

RL-Driven Multi-Source Retrieval

The retrieval agent operates over an 8-dimensional action space, spanning specialized and generalist search tools. Each search action is dynamically selected via a policy network conditioned on claim category, factuality type, recentness, and prior search state. Customized query generation and source selection ensure both comprehensive coverage and focused refutation-seeking, mitigating confirmation bias and overfitting to superficial signals.

Evidence Evaluation

Each retrieved document is chunked and processed through LLM-based parsing pipelines to extract relevant statements, which are annotated with stance, authority, and provenance metadata. The evaluation agent then performs stance-specific attention, quality-weighted evidence aggregation, and joint prediction of veracity and reliability, leveraging both inter-evidence relationships and claim–evidence interactions.

Figure 3: Detail of Evaluation Network.

Neural Tree Aggregation

Aggregation proceeds by fusing each node's local veracity/reliability with those of its subtree, via a sequence of self- and cross-attention mechanisms, mean-pooling, and feature integration heads. The aggregator is trained end-to-end with a combination of supervised (root node) and imitation (non-root) losses, using LLM-judged soft targets for both veracity and reliability.

Figure 4: Detail of Tree Aggregation Network.

Tree expansion is governed by per-node reliability thresholds; claims with insufficient evidence are decomposed via LLMs into fine-grained, importance-weighted sub-claims, recursively enabling divide-and-conquer verification.

Empirical Evaluation

Robustness to Adversarial and Realistic Claims

Evaluations are carried out on LIAR, PolitiFact, and Check-COVID benchmarks, in addition to the AdvFact synthetic adversarial dataset constructed using FakeGPT and PoisonedRAG techniques to simulate paraphrased and retrieval-poisoned misinformation.

Results indicate consistent improvement (4–24 percentage points, depending on dataset and LLM backbone) over baselines such as Direct prompting, TELLER, STEEL, AFaCTA, and AdSent. On AdvFact, ToE surpasses all comparison methods, with accuracy gains especially pronounced against retrieval poisoning adversarial attacks. The reliability of ToE extends to both large (DeepSeek-V3.2) and smaller (gpt-oss-20b) LLMs, indicating generalizability and limited dependence on specific model capabilities.

Search Strategy Adaptivity

Analysis of retrieval agent action distributions reveals category-aware adaptation: scientific domains induce frequent arXiv queries, political/health claims trigger fact-checking source searches, while celebrity claims stimulate social media retrieval. Each tool contributes complementary evidence; ablation identifies fact-checking sites as most essential for true-claim verification, and removal of any major source substantially degrades accuracy.

Figure 5: Heatmap of Action Distribution.

Implications and Future Directions

Practically, ToE offers interpretable and robust claim verification resilient to GEO poisoning, providing a viable blueprint for integrating LLMs within risk-sensitive information pipelines—news aggregation, search, and platform moderation. Theoretically, the POMDP-based formalization and associated performance bounds open avenues for more sophisticated active information-seeking under uncertainty, with extensibility to higher-dimensional or online evolving evidence spaces.

Future research should consider:

• Extending ToE to multi-hop question answering and temporal fact tracking.
• Investigating richer uncertainty quantification and error attribution within argument trees.
• Optimizing tree structure induction using differentiable or search-based sub-claim decomposition.
• Studying human-in-the-loop interaction for post-hoc verification and evidence chain auditing.

Conclusion

The ToE framework demonstrates that explainable, hierarchical claim verification using RL-optimized dynamic retrieval and neural evidence aggregation yields robust, traceable judgments across adversarial and in-the-wild misinformation settings. This approach constitutes an important advancement toward reliable fact-checking under open-world, adversarial conditions (2606.27736).