Beyond Static Benchmarks: Synthesizing Harmful Content via Persona-based Simulation for Robust Evaluation

Abstract: Static benchmarks for harmful content detection face limitations in scalability and diversity, and may also be affected by contamination from web-scale pre-training corpora. To address these issues, we propose a framework for synthesizing harmful content, leveraging persona-guided LLM agents. Our approach constructs two-dimensional user personas by integrating demographic identities and topical interests with situational harmful strategies, enabling the simulation of diverse and contextually grounded harmful interactions. We evaluate the framework along three dimensions: harmfulness, challenge level, and diversity. Both human and LLM-based evaluations confirm that our framework achieves a high harmful generation success rate. Experiments across multiple detection systems reveal that our synthetic scenarios are more challenging to detect than those in existing benchmarks. Furthermore, a multi-faceted analysis confirms that our approach achieves linguistic and topical diversity comparable to human-curated datasets, establishing our framework as an effective tool for robust stress-testing of harmful content detection systems.

• The paper presents a persona-augmented simulation framework that dynamically synthesizes harmful content, offering a scalable alternative to static benchmarks.
• The methodology combines intrinsic and extrinsic persona elements to simulate realistic user behaviors, achieving a 96.8% success rate in generating harmful content.
• The evaluation reveals significant drops in safety classifier detection rates, highlighting vulnerabilities in current harmful content detection models.

Persona-Guided Synthesis of Harmful Content for Robust Evaluation

Motivation and Limitations of Static Benchmarks

Evaluation of harmful content detection models has traditionally relied on manually curated or platform-harvested static benchmarks. While these resources were foundational, they suffer from poor scalability, limited topical and stylistic diversity, and contamination from pre-training corpora, undermining their validity for safety-critical LLM applications. These constraints render detection models vulnerable to rare, subtle, or emergent threats that are systematically unrepresented in conventional benchmarks.

Persona-Based Synthetic Generation Framework

The presented work introduces a persona-augmented simulation framework for the dynamic synthesis of harmful content (2604.17020). The key innovation is the generation of user personas—structured profiles that inject both identity and behavioral diversity into LLM agents. Each persona encodes:

• Intrinsic aspects: Demographics (e.g., username, biography, subreddit interests, interaction style) to ground the agent in a coherent, human-like profile.
• Extrinsic aspects: Interaction strategies (e.g., trolling strategies, abusive categories) that condition the communicative intent and harmful style of generated content.

Persona-guided agents are deployed into discussion threads sampled from the Pushshift Reddit corpus, where they generate contextually aligned harmful comments. By independently pairing intrinsic and extrinsic aspects, the framework enables compositionally rich and controllable generation of synthetic scenarios, supporting the discovery of detection system blind spots.

Evaluation and Analysis

Harmfulness and Challenge Assessment

Human annotation achieves $\kappa=0.70$ (substantial agreement), with annotator majority accuracy at 96%, establishing that generated comments are perceived as distinctly harmful. LLM-based evaluation by GPT-4o and Claude-3.5 Sonnet yields an average harmfulness generation success rate of 96.8%, a +6.4% improvement via persona-augmentation compared to context-only conditions.

Detection rates for four representative safety classifiers—including OpenAI Moderation API, Perspective API, and LlamaGuard variants—are substantially lower for persona-based synthetic scenarios than static benchmarks. In the trolling-oriented setting, average detection accuracy drops to 13.5% (vs. 28.7% on best static baseline); for the CADD-based explicit-abuse setting, detection drops to 31.1% (vs. 63.2% on CADD). This demonstrates that the synthetic framework can consistently generate hard-to-detect harmful content, exposing classifier blind spots not apparent from standard datasets.

Figure 1: Detection patterns and misses visualized via t-SNE for four classifiers on the generated harmful content.

Diversity Metrics

The analysis integrates embedding-based and token-based evaluations. Pairwise embedding distance and convex hull area in t-SNE space reveal a broader and less concentrated distribution for persona-based scenarios against all static benchmarks.

Figure 2: t-SNE embedding plots show that synthetic data surpasses static benchmarks in coverage and diversity.

Linguistic diversity is further validated via lower Self-BLEU, higher TTR, and greater vocabulary size; categorical diversity is measured by increased Shannon entropy over computed trolling/harmful types. Gains are consistent across all LLM generators (GPT-4o, Llama-3.1 70B, DeepSeek-Llama 70B) and extend to non-English contexts.

Ablations indicate that intrinsic and extrinsic persona components each independently enhance diversity—maximal effect requires their joint conditioning.

Figure 3: Intrinsic-only, extrinsic-only, and combined persona ablations all expand the embedding space beyond context-only baselines.

User-Type-Centric and Community Analysis

Subreddit engagement varies systematically with the synthetic user type (newcomer, regular user, longtime user), supporting ecological plausibility in profile simulation.

Figure 4: User type conditions the distribution over subreddit categories, controlling forum exposure in synthetic scenarios.

Comment style and vocabulary length also show controlled variation by user type. Case studies highlight how compositional persona aspects result in both thematic and strategic variance in generated harms.

Practical and Theoretical Implications

The framework establishes that persona-augmented synthetic data generation is an effective adversarial stress test for harmful content detection models. By producing previously unobserved, highly diverse, and contextually plausible harms that evade existing classifiers, it exposes fundamental limits in current safety models and provides a substrate for systematic robustness evaluation.

The modular composition of persona aspects facilitates parameteric evaluation of detection models' generalization to rare behaviors or user profiles unseen during training. The integration of agent-based simulation connects with research trajectories in LLM agent social modeling but is adapted here as a safety evaluation strategy.

Methodologically, the approach motivates further work in dynamic, continually updated benchmarks that match evolving online harms, mitigating contamination and static bias in web-scale benchmarks.

Limitations and Future Directions

The current work focuses on single-turn harmful comment synthesis. Multi-turn dialog simulation and the automatic discovery of novel harm strategies are identified as next steps. The generation process currently leverages safety-aligned LLMs, partially constraining the explicitness of produced harms and reflecting an evaluation focus (i.e., evasion rather than overt abrogation).

The strategic flexibility and compositional nature of the persona-based framework could support dynamic adversarial evaluation paradigms, continual updating of safety evaluation protocols, and the study of detection in multilingual or cross-community scenarios.

Conclusion

Persona-based simulation with two-dimensional profile-strategy composition produces synthetic harmful content that is more diverse, contextually plausible, and challenging to detect than static benchmarks. This introduces a scalable and flexible tool for stress-testing detection models, supporting advances in model robustness, risk modeling, and future-proofing of safety evaluation protocols. The findings advocate the replacement or augmentation of static benchmarks with dynamic, persona-augmented simulation frameworks for critical evaluation of both LLM-based and traditional detection architectures.