Fraud-Aware Selective CoT Distillation

• The paper introduces a novel fraud-aware selective CoT distillation mechanism that autonomously generates and selects multi-hop reasoning chains to reveal fraud cues in text-attributed graphs.
• The methodology combines semantic and structural cues through an integrated LLM-GNN pipeline, achieving up to +8.8% improvement in AUPRC over leading baseline models.
• The approach employs asymmetric co-training and caching strategies to reduce LLM calls and scale batch sizes efficiently, enabling significant computational speedups.

Fraud-aware selective Chain-of-Thought (CoT) distillation is a targeted mechanism developed to enhance graph-based fraud detection in text-attributed graphs (TAGs) by autonomously generating and selecting diverse, semantically-relevant reasoning paths. Integrated within the FraudCoT framework, it addresses the limitations of predefined prompting and decoupled LLM-GNN training pipelines by aligning semantic and structural cues from textual and relational data. This process involves autonomous LLM generation of multi-hop reasoning chains, their selective distillation based on fraud-relevance and diversity, and their integration into node representations for downstream graph neural network (GNN) classification—enabling state-of-the-art detection performance and scalable end-to-end optimization (Li et al., 30 Jan 2026).

1. Problem Formulation and Notation

Fraud-aware selective CoT distillation operates on a text-attributed graph (TAG) $G = (V, E, X)$, where $V$ is the set of nodes (e.g., reviews, transactions, users), $E \subseteq V \times V$ represents binary relations, and $X = \{x_v\,|\,v \in V\}$ comprises raw textual attributes. Each node $v$ is assigned a binary fraud label $y_v \in \{0, 1\}$, and $A \in \{0, 1\}^{|V|\times|V|}$ denotes the adjacency matrix, with $N(v)$ defining the set of neighbors of $v$. The objective is to learn a scoring function $f_\theta(v)$ approximating $P(y_v=1\,|\,x_v, N(v), \ldots)$ for accurate node-level fraud detection (Li et al., 30 Jan 2026).

2. Autonomous Chain-of-Thought Generation

The distillation process begins with teacher LLM-based generation of graph-aware reasoning chains for selected nodes. For each distillation node $v_i$, a composite prompt $p_i$ is constructed by concatenating its raw text $x_i$ and textual descriptions of its 1-hop or multi-hop neighbors $\{x_j \mid j \in N(v_i)\}$. The teacher LLM $T$ receives $p_i$ and samples $S$ chain-of-thoughts:

$r_i^{T,s} = T(p_i), \quad s = 1, \ldots, S.$

Each $r_i^{T,s}$ is a free-form path that references both local semantic features and relational (neighbor) signals, aiming to uncover multi-hop fraud cues embedded in the graph structure (Li et al., 30 Jan 2026).

3. Selective Distillation: Relevance Scoring and Diversity Enforcement

A fraud-aware CoT scoring function $s(r;G)$ is proposed to select high-value chains:

$$s(r;G) = \alpha\,\text{sim}(\text{Enc}_\text{text}(r), \text{Enc}_\text{text}(x_i)) + (1-\alpha)\, \sum_{j\in N(v_i)} \text{sim}(\text{Enc}_\text{text}(r), \text{Enc}_\text{text}(x_j)),$$

where $\text{sim}(\cdot,\cdot)$ is cosine similarity and $\text{Enc}_\text{text}$ is a lightweight encoder (e.g., BERT-tiny), balancing semantic relevance and structural alignment. Scores are normalized to $[0,1]$ across samples.

To ensure that distilled CoTs are both fraud-relevant and diverse, the procedure selects:

• $S_{\text{pos}}$: the top-$k$ CoTs with scores $\ge \tau_{\text{pos}}$,
• $S_{\text{neg}}$: the bottom-$k$ CoTs with scores $\le \tau_{\text{neg}}$, retaining only those with pairwise Levenshtein edit distance $\ge D$ to enforce diversity.

The student model $S$ is distilled by minimizing:

$$\mathcal{L}_\text{distill} = \mathbb{E}_{r\in P}[-\log P_S(r|p_i)] + \lambda\,\mathbb{E}_{r\in N}[-\log(1-P_S(r|p_i))],$$

where $P$ and $N$ are sets of positive and negative chains, and $\lambda$ is an unlikelihood loss coefficient. LoRA (Low-Rank Adaptation) adapters with rank $r=8$–$16$ are employed for efficient parameterization during student finetuning. Empirically, $U \approx 300$ nodes, $S=5$ samples, $k=2$–3, and $\lambda \approx 100$ are optimal (Li et al., 30 Jan 2026).

4. Integration of Distilled CoTs into GNN Representation

After student finetuning, a distilled CoT is generated for each node $v$ as $r_v = S(x_v, x_{N(v)})$ and concatenated with the raw text to yield $x'_v = [x_v \oplus r_v]$. This extended textual attribute is processed by an LLM-style encoder $\text{Enc}_\theta$ to obtain initial node embeddings:

$h_v^{(0)} = \text{Enc}_\theta(x'_v)$

These enriched embeddings encode both direct and multi-hop semantic-structural fraud cues, providing a more informative input for the downstream GNN (Li et al., 30 Jan 2026).

5. Asymmetric LLM–GNN Co-Training and Efficiency

A key component is the asymmetric co-training pipeline. In each batch $B$, only target nodes $v_i$ are freshly encoded:

$h_i^{(0)} = \text{Enc}_\theta(x'_i)$

while neighbor nodes $j \in N(v_i)$ utilize cached embeddings $\bar h_j^{(0)}$ computed once at initialization. Message passing employs $L$ heterogeneous GraphSAGE layers:

$$h_i^{(l+1)} = \sigma\left( W_r^{(l)}\cdot \text{mean}\left( \{ h_i^{(l)} \} \cup \{ h_j^{(l)} : j \in N_r(v_i) \} \right) \right)$$

The total loss is a weighted sum of binary cross-entropy node classification and distillation loss:

$$\mathcal{L} = \alpha\,\mathcal{L}_\text{GNN} + (1-\alpha)\,\mathcal{L}_\text{distill}$$

Typically, $\alpha=0.8$. Only target nodes propagate gradients through $\text{Enc}_\theta$, greatly reducing backpropagation cost.

In terms of efficiency, the asymmetric strategy decreases per-epoch LLM calls from $O(NK^HL(TD^2+T^2D))$ (naive) to $O(N)$, with a one-time $O(N)$ cache build. Using the DigitalMusic dataset, this yields epoch time reduction from $58{,}639$ s (naive) to $55$ s (asymmetric), corresponding to a $1{,}066 \times$ speedup, and permits batch size scaling from $2$ to $256$ (Li et al., 30 Jan 2026).

6. Empirical Evaluation and Baselines

FraudCoT, incorporating fraud-aware selective CoT distillation, is experimentally validated on InstantVideo (37,126 nodes, 9.9M edges), DigitalMusic (64,706 nodes, 7.7M edges), and PromotionAbuse (371K nodes, 1.3M edges). Comparisons are drawn against:

• GNNs: GraphSAGE, HGT, ConsisGAD, PMP, GAAP
• Pure LLMs: Qwen-8B, InstructGLM
• Graph-enhanced LLMs: LLaGA, GraphGPT, HiGPT
• LLM-enhanced GNNs: TAPE, FLAG

Evaluation metrics include Macro-F1, AUROC, and AUPRC (using scikit-learn implementations). FraudCoT achieves up to $+8.8\%$ improvement in AUPRC over the best baseline (PromotionAbuse), with consistent performance gains across all public and industrial dataset splits. Ablation studies confirm that both the selective distillation and asymmetric co-training components are essential for optimal results (Li et al., 30 Jan 2026).

Dataset	Nodes	Edges	Best AUPRC Gain
InstantVideo	37,126	9.9M	up to +8.8%
DigitalMusic	64,706	7.7M	up to +8.8%
PromotionAbuse	371,000	1.3M	up to +8.8%

7. Implementation Strategies and Practical Considerations

Implementation guidelines include:

• Use LoRA adapters with rank $r=8$–$16$.
• Set unlikelihood loss $\lambda \approx 100$.
• Sample $K=10$ neighbors, with hop number $H=2$.
• Cache node embeddings on CPU or host memory to minimize GPU LLM calls.
• Apply early stopping based on validation AUROC; batch sizes $128$–$256$ provide optimal throughput.
• Precompute all text encodings for CoT scoring to accelerate distillation selection.

A plausible implication is that this design allows practitioners to scale LLM-enhanced GNNs to industrial TAGs with both superior predictive performance and orders-of-magnitude throughput improvements, without requiring expensive end-to-end LLM inference on all neighbors during training (Li et al., 30 Jan 2026).