TAGFN: Graph Benchmark for Fake News

• TAGFN is a large-scale, real-world text-attributed graph dataset designed to evaluate fake news detection by modeling news propagation and textual user interactions.
• It represents news items as propagation graphs where the root contains the news text and child nodes capture user posts, effectively analyzing diffusion patterns.
• The benchmark integrates diverse data sources like Politifact, GossipCop, and Fakeddit, and supports both LLM prompting and supervised GNNs for outlier detection.

TAGFN is a large-scale, real-world text-attributed graph dataset for outlier detection, specifically fake news detection, introduced to support rigorous evaluation of both traditional and LLM-based graph outlier detection methods in settings where large-scale, realistic, and well-annotated benchmarks have been scarce (Liu et al., 26 Nov 2025). It represents each news item as a propagation graph with textual node attributes, enabling graph-level prediction of whether the item is real or fake, and is intended both as a benchmark and as a resource for developing misinformation detection capabilities in LLMs (Liu et al., 26 Nov 2025).

1. Research setting and problem formulation

TAGFN situates fake-news detection within graph outlier detection. Each instance is a graph $\mathcal{G}_i$, and the prediction target is a binary graph label $y_i\in\{0,1\}$ indicating real $(0)$ or fake $(1)$ (Liu et al., 26 Nov 2025). This formulation places emphasis on joint reasoning over content and propagation rather than on text classification in isolation.

The dataset is motivated by the observation that the application of LLMs to graph outlier detection remains significantly underexplored, especially for fake news detection, and that progress has been constrained by the lack of large-scale, realistic, and well-annotated benchmarks (Liu et al., 26 Nov 2025). TAGFN is therefore designed as a text-attributed graph benchmark rather than as a corpus of standalone documents.

A central design decision is to model fake-news instances as propagation graphs rooted at the news item itself. In practice, most graphs form small trees or cascades rooted at the news node (Liu et al., 26 Nov 2025). This suggests that the dataset emphasizes diffusion structure and local interaction patterns that arise when users share, comment on, or retweet an item.

2. Graph schema and textual attribution

Each data instance is a propagation graph

$$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$

built around one news item (Liu et al., 26 Nov 2025). The root node $v_0$ represents the news article itself and is attributed with the raw news text $\mathbf{x}_0$. For $j>0$, node $v_j$ represents a user who shared, commented on, or retweeted the news, and each such node is attributed with a concatenation of up to 200 of the user’s most recent posts, denoted $\mathbf{x}_j$ (Liu et al., 26 Nov 2025).

Edges encode direct propagation. An edge $y_i\in\{0,1\}$0 is included if user $y_i\in\{0,1\}$1 directly propagates the information from $y_i\in\{0,1\}$2, for example through a retweet or a reply (Liu et al., 26 Nov 2025). The graph therefore captures a diffusion cascade whose semantics are operational rather than purely social: the adjacency relation encodes information flow for the specific news item.

The textual attributes are raw un-masked text for both news and user posts, with personal IDs masked, and an optional Unix timestamp $y_i\in\{0,1\}$3 may be associated with each node (Liu et al., 26 Nov 2025). For prompting with LLMs, the preprocessing is intentionally constrained by context-window considerations: user posts per node are limited in total to 500 characters, and at most 30 users are included in any prompt; more specifically, for LLM prompting only the top 30 child nodes and 500 characters per node are used (Liu et al., 26 Nov 2025).

3. Data sources, labeling, and corpus scale

TAGFN aggregates three subsets: Politifact, GossipCop, and Fakeddit (Liu et al., 26 Nov 2025). Politifact and GossipCop are sourced from FakeNewsNet and UPFD, but the raw text is re-gathered via news URLs and user timelines rather than retaining only BERT or word2vec embeddings (Liu et al., 26 Nov 2025). Fakeddit is sourced from r/fakeddit; full news posts and comment trees are collected via the Pushshift API, bot accounts are filtered via BotRank, and instances with zero comments are removed (Liu et al., 26 Nov 2025).

Ground-truth labels are inherited from the original sources. Politifact uses human-verified labels, GossipCop uses consensus labels, and Fakeddit’s fine-grained annotations are collapsed to binary with fake $y_i\in\{0,1\}$4 and real $y_i\in\{0,1\}$5 (Liu et al., 26 Nov 2025). Public splits are released consistently with the prior benchmarks (Liu et al., 26 Nov 2025).

Subset	Scale	Splits
Politifact	41,054 nodes; 40,740 edges; 314 graphs; Avg. $y_i\in\{0,1\}$6; Fake 50.0%	Train 62; Val 31; Test 221
GossipCop	314,262 nodes; 308,798 edges; 5,464 graphs; Avg. $y_i\in\{0,1\}$7; Fake 50.0%	Train 1,092; Val 546; Test 3,826
Fakeddit	7,249,803 nodes; 6,683,699 edges; 566,104 graphs; Avg. $y_i\in\{0,1\}$8; Fake 59.6%	Train 467,538; Val 49,186; Test 49,380

The scale disparity across the three subsets is methodologically important. Politifact contains relatively few but larger graphs on average, GossipCop is intermediate, and Fakeddit is substantially larger in graph count but much smaller in average graph size (Liu et al., 26 Nov 2025). A plausible implication is that these subsets probe different operating regimes for graph-text models: few large cascades, many medium cascades, and very many small cascades.

4. Representations, prompting, and supervised learning protocol

TAGFN supports two broad representation pathways. First, raw text can be directly fed into LLM prompts for zero-shot or few-shot in-context learning. Second, text can be embedded via foundation models such as Qwen3-Embedding-8B and then supplied to graph neural architectures (Liu et al., 26 Nov 2025).

For the supervised GNN baselines, node embeddings $y_i\in\{0,1\}$9 are extracted from an LLM embedding API and then propagated with GraphSAGE (Liu et al., 26 Nov 2025). The adjacency structure is used through the update

$(0)$0

No additional handcrafted structural statistics, such as degree or clustering coefficient, are used in the baselines (Liu et al., 26 Nov 2025). The baseline therefore isolates the contribution of text embeddings and neighborhood aggregation without adding manual graph descriptors.

LLM prompts follow a two-part structure: a system message describing the task and a user message encoding node texts and edge lists (Liu et al., 26 Nov 2025). No end-to-end fine-tuning of LLMs is reported in the benchmark; instead, the experiments use off-the-shelf LLM embedders and structured prompting (Liu et al., 26 Nov 2025). This distinction is important because TAGFN is presented as enabling fine-tuning, but the reported results measure prompting and embedding-based graph supervision rather than parameter-updated LLM adaptation.

The supervised Emb+GNN baseline is trained with cross-entropy loss

$(0)$1

with standard $(0)$2-regularization on GNN weights (Liu et al., 26 Nov 2025).

5. Evaluation protocol and baseline performance

The benchmark reports Precision, Recall, F1-score, Accuracy, and, when available, AUC (Liu et al., 26 Nov 2025). Precision and Recall are defined in the standard way from $(0)$3, $(0)$4, $(0)$5, and $(0)$6, Accuracy is

$(0)$7

and AUC measures the trade-off between true positive rate and false positive rate across thresholds (Liu et al., 26 Nov 2025).

The traditional graph-based baseline, denoted Emb+GNN, embeds each node with Qwen3-Embedding-8B and then trains GraphSAGE on the training graphs (Liu et al., 26 Nov 2025). Its reported performance is ACC $(0)$8, F1 $(0)$9 on Politifact; ACC $(1)$0, F1 $(1)$1 on GossipCop; and ACC $(1)$2, F1 $(1)$3 on Fakeddit (Liu et al., 26 Nov 2025).

Prompt-based results using Qwen3-8B show substantial variation by prompting strategy. On Politifact, Zero-Shot yields ACC/F1 of $(1)$4, Reasoning yields $(1)$5, One-Shot yields $(1)$6, Two-Shot yields $(1)$7, and Three-Shot yields $(1)$8 (Liu et al., 26 Nov 2025). On GossipCop, the corresponding values are $(1)$9, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$0, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$1, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$2, and $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$3 (Liu et al., 26 Nov 2025). On Fakeddit, they are $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$4, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$5, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$6, $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$7, and $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$8 (Liu et al., 26 Nov 2025).

The comparison across LLMs on Politifact further differentiates closed and open models under zero-shot and one-shot settings. Qwen3-8B reaches $$\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathcal{T}),$$9 in zero-shot and $v_0$0 in one-shot; GPT-4.1-nano reaches $v_0$1 and $v_0$2; GPT-4.1-mini reaches $v_0$3 and $v_0$4; GPT-4.1 reaches $v_0$5 and $v_0$6; GPT-5 reaches $v_0$7 and $v_0$8; and O3 reaches $v_0$9 and $\mathbf{x}_0$0 (Liu et al., 26 Nov 2025).

These results support three observations stated in the benchmark. In-context learning and reasoning substantially boost small LLM performance; supervised Emb+GNN remains state-of-the-art on large and balanced subsets; and context length can hurt performance beyond one-shot, especially on larger graphs (Liu et al., 26 Nov 2025). The last point bears directly on prompt design: increasing exemplars is not monotonically beneficial when graph serialization consumes a large fraction of the available context budget.

6. Interpretive significance, limitations, and open directions

The benchmark’s principal empirical takeaway is that the synergy of propagation structure and text is critical: removing edges or user posts in ablation drops ACC by up to 15 points (Liu et al., 26 Nov 2025). This indicates that TAGFN is not merely a text dataset rendered in graph form; its structure materially affects prediction quality.

A second takeaway is that few-shot prompting can approach supervised GNN performance on small graphs, specifically Politifact, but degrades on larger cascades (Liu et al., 26 Nov 2025). A plausible implication is that prompt-based graph reasoning remains sensitive to serialization length and to the combinatorial growth of diffusion context. By contrast, Embedding + GNN remains a strong baseline, which the paper interprets as evidence for the value of explicit supervision (Liu et al., 26 Nov 2025).

Several limitations are explicit. The graphs are treated as static, and naive inclusion of timestamps had no benefit, so temporal modeling remains unexplored (Liu et al., 26 Nov 2025). Large open-source models with more than 10B parameters and other LLM families, including Claude, Gemini, and Mistral, were not benchmarked because of resource constraints (Liu et al., 26 Nov 2025). These limitations clarify two common misconceptions: TAGFN is not yet a temporal misinformation benchmark, and it is not a comprehensive cross-family evaluation of contemporary LLMs.

The future directions identified for TAGFN include dynamic graph modeling that leverages node-level timestamps, end-to-end fine-tuning of LLMs on TAGFN via parameter-efficient techniques such as LoRA and adapters, synthetic out-of-distribution augmentation for robustness in misinformation detection, and extension to multimodal propagation graphs involving images, videos, and cross-platform information flow (Liu et al., 26 Nov 2025). In this sense, TAGFN functions both as a benchmark for current methods and as infrastructure for later work on trustworthy AI, graph outlier detection, and the integration of LLMs with structured data (Liu et al., 26 Nov 2025).