TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns

Abstract: LLM-driven (LLM-driven) social bots pose a growing threat to online discourse by generating human-like content that evades conventional detection. Existing methods suffer from limited detection accuracy due to overreliance on single-modality signals, insufficient sensitivity to the specific generative patterns of Artificial Intelligence-Generated Content (AIGC), and a failure to adequately model the interplay between linguistic patterns and behavioral dynamics. To address these limitations, we propose TRACE-Bot, a unified dual-channel framework that jointly models implicit semantic representations and AIGC-enhanced behavioral patterns. TRACE-Bot constructs fine-grained representations from heterogeneous sources, including personal information data, interaction behavior data and tweet data. A dual-channel architecture captures linguistic representations via a pretrained LLM and behavioral irregularities via multidimensional activity features augmented with signals from state-of-the-art (SOTA) AIGC detectors. The fused representations are then classified through a lightweight prediction head. Experiments on two public LLM-driven social bot datasets demonstrate SOTA performance, achieving accuracies of 98.46% and 97.50%, respectively. The results further indicate strong robustness against advanced bot strategies, highlighting the effectiveness of jointly leveraging implicit semantic representations and AIGC-enhanced behavioral patterns for emerging LLM-driven social bot detection.

• The paper introduces a dual-channel framework that fuses GPT-2 based semantic encoding with AIGC-enhanced behavioral features to detect LLM-driven social bots.
• It demonstrates superior detection accuracy over 11 baselines using rigorous experiments with metrics such as F1-scores and Adjusted Rand Index.
• It highlights practical benefits in label efficiency, robustness to class imbalance, and scalability for ensuring online discourse integrity.

TRACE-Bot: Dual-Channel Detection of Emerging LLM-Driven Social Bots

Introduction and Motivation

The proliferation of social bots leveraging LLMs presents critical challenges for the integrity of online discourse. These bots are capable of generating highly human-like content, dynamically adapting linguistic style and behavioral strategies to bypass traditional detection techniques. Prior approaches, often predicated on unimodal signals or loosely fused multimodal features, have demonstrated limited efficacy against LLM-driven bots due to their inability to jointly model implicit semantics and behavioral anomalies characteristic of Artificial Intelligence-Generated Content (AIGC).

The paper "TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns" (2604.02147) introduces TRACE-Bot, a unified dual-channel detection framework that addresses key limitations of prior work—insensitivity to generative AIGC signatures, weak cross-modal integration, and the inability to capture nuanced interplay between linguistic and behavioral artifacts produced by LLMs. TRACE-Bot proposes a systematic pipeline for feature extraction, fusion, and classification tailored for emerging LLM-driven social bots.

Architecture and Methodology

TRACE-Bot consists of four core modules: Data Preparation, Feature Processing, Feature Fusion, and Detection. The system leverages user profile metadata, temporal interaction sequences, and tweet content, integrating these heterogeneous sources via a dual-channel architecture.

Figure 1: The overall TRACE-Bot framework integrating dual-channel encoding for implicit semantics and AIGC-enhanced behavior.

Data Preparation and Feature Extraction

The framework is designed around two LLM-driven social bot datasets, Fox8-23 and BotSim-24, comprising real-world and simulated bot/human accounts with rich multimodal signals. Data modalities utilized include:

• Personal Information: Profile meta-attributes such as usernames, bios, images, verification status, and locale.
• Interaction Behavior: Chronologically ordered sequences of tweet types (original, retweet, reply) transformed into symbolic strings and compressed to quantify repetition and structure, characteristic of automation.
• Tweet Content: Each tweet is scored using state-of-the-art AIGC detectors (Fast DetectGPT, GLTR), with summary statistics aggregated as probabilistic features indicating the prevalence and confidence of machine-generated text.

Dual-Channel Feature Fusion

TRACE-Bot's core innovation lies in its dual-channel fusion mechanism:

• Textual Channel: GPT-2 encodes all relevant textual inputs (profile fields and tweet contents), producing contextualized semantic embeddings. Mean pooling with attention masks is adopted to generate robust user-level representations, capturing stylistic artifacts and context coherence anomalies indicative of LLM generation.
• Behavioral Channel: A tailored MLP processes numerical/behavioral features, including interaction statistics and AIGC detection metrics. Features are standardized and subjected to ReLU activation and Dropout for robust nonlinear transformation, enhancing the model's sensitivity to subtle behavioral irregularities.

Concatenated channel outputs serve as input to a lightweight classification MLP for final prediction, with training optimized via class-weighted cross-entropy to handle real-world class imbalance.

Empirical Evaluation

The efficacy of TRACE-Bot is rigorously validated against 11 representative baselines, including traditional ML, deep learning, GNN-based, and LLM-based detectors. Experiments employ the Fox8-23 and BotSim-24 datasets, each partitioned into training, validation, and test splits, with careful control for class balance and sample representativeness.

Detection Results

TRACE-Bot establishes new SOTA performance across all standard metrics (Accuracy, Precision, Recall, F1-Score) on both datasets:

• Fox8-23: 98.46% Accuracy, 98.25% Precision, 98.68% Recall, 98.47% F1.
• BotSim-24: 97.50% Accuracy, 95.67% Precision, 99.50% Recall, 97.55% F1.

These results consistently outperform baselines, including recent LLM-based and multi-modal fusion approaches, particularly excelling in reducing false positives—a primary challenge in human-like bot detection.

Ablation and Modality Studies

Ablation removes either channel or input modality to assess marginal contribution:

• Excluding the textual channel produces a 2.82 drop in F1; removing the behavioral channel yields a more significant F1 collapse (6.07), highlighting the primacy of behavioral features in suppressing precision loss.
• Modality ablation reveals the necessity of profile information for optimal discrimination, while fusion of all three (profile, behavior, tweet content) is essential for performance and generalizability, especially on more adversarial datasets (e.g., BotSim-24).

Representation Analysis

t-SNE visualizations demonstrate superior class separation and cluster compactness in TRACE-Bot's latent space compared to strong baselines.

Figure 2: t-SNE visualization of TRACE-Bot's learned representations on Fox8-23, with clear separation between human and bot clusters.

Figure 3: t-SNE visualization on BotSim-24 shows highly compact, non-overlapping clusters for humans and LLM-driven bots.

Quantitatively, TRACE-Bot attains ARI and Silhouette Index values significantly above other state-of-the-art techniques, confirming the discriminative capacity of its multimodal representation learning.

Robustness and Hyperparameter Analysis

TRACE-Bot maintains robust performance under variable training label proportions, label imbalance, and across a broad hyperparameter search space. In particular, it saturates near peak F1 (≥0.96) even with only 30-40% labeled data, demonstrating efficient use of annotation resources and strong generalization. The model is not overly sensitive to learning rate choices for either GPT-2 or MLP channels, with performance sweet spots spanning several orders of magnitude.

Figure 4: TRACE-Bot F1-Score as a function of labeled training data fraction, highlighting extreme label efficiency.

Figure 5: Robustness to training set size and bot/human imbalance, with stable performance even under extreme shifts.

Figure 6: Hyperparameter heatmap for varying learning rates in GPT-2 and MLP channels; broad region of strong performance.

Model Design Justification

Comparison of different PLMs and prompt strategies for the semantic channel indicates that direct encoding of concatenated raw-text attributes in GPT-2 yields the highest F1, outperforming pre-prompted templates and alternative encoders (e.g., BERT, RoBERTa).

Qualitative Analysis

A representative case from Fox8-23 is detailed to delineate the synergistic contribution of both channels:

Figure 7: Case study showing bot detection confidence from TRACE-Bot; both textual and behavioral channels independently and jointly flag the account as bot, underscoring the importance of multimodal fusion.

The example showcases both LLM telltale linguistic artifacts and abnormal behavioral signatures, each independently sufficient, and overwhelmingly compelling when considered in fusion.

Practical and Theoretical Implications

TRACE-Bot advances the state-of-the-art in social bot detection by introducing the first framework to jointly model implicit LLM-induced semantic representations and AIGC-enhanced behavioral signals. Its high precision/recall suggests suitability for deployment in high-stakes moderation and platform integrity systems, where high false positive rates are operationally unacceptable.

The theoretical contribution lies in validating dual-channel fusion as a superior paradigm compared to unimodal or naïvely fused approaches, particularly as generative model sophistication (and cross-modality deception) continues to accelerate. By incorporating AIGC detection as a feature source, TRACE-Bot lays a foundation for joint reasoning over generative and behavioral footprints, which will be essential as adversarial bot strategies evolve.

Limitations and Future Directions

The current framework is bounded by its English-centric datasets and Twitter/X domain specificity. Generalization to non-English platforms and traditional rule-based bots remains an open challenge. Future research directions include cross-lingual/cross-platform adaptation, extension to richer media types (image, video, audio), and integrating more robust synthetic media detectors for next-generation multimodal bot detection.

Conclusion

TRACE-Bot delivers a highly effective, robust, and label-efficient solution for LLM-driven social bot detection via a discriminative dual-channel architecture. Its demonstrated superiority across diverse benchmarks, supported by detailed ablation and representation analysis, establishes a new practical and methodological benchmark for future social bot detection research and development.

---

References:

Full details in "TRACE-Bot: Detecting Emerging LLM-Driven Social Bots via Implicit Semantic Representations and AIGC-Enhanced Behavioral Patterns" (2604.02147).