GuardEval: Unified LLM Moderation Benchmark

• GuardEval is a unified dataset and evaluation framework addressing LLM moderation challenges such as implicit bias, nuanced harms, and adversarial prompts.
• It aggregates thirteen safety datasets into a detailed taxonomy of 106 subcategories, facilitating granular detection and analysis of harmful content.
• Accompanied by the QLoRA-fine-tuned GemmaGuard model, GuardEval demonstrates significant improvements in both prompt and response moderation performance.

GuardEval is a unified multi-perspective benchmark dataset and evaluation framework targeting the safety, fairness, and robustness of LLM content moderators. Designed to address limitations in existing evaluations—especially the detection of nuanced bias, implicit offensiveness, and adversarial prompt handling—GuardEval aggregates community, adversarial, and normative perspectives across a broad taxonomy of harms. It is accompanied by GemmaGuard (GGuard), a QLoRA-fine-tuned variant of Gemma3-12B that sets new benchmarks for granular moderation performance (Machlovi et al., 22 Dec 2025).

1. Motivation and Foundational Challenges

The pervasive deployment of LLMs has accentuated the inadequacy of conventional moderation systems, with three primary challenge domains:

1. Subtle and Implicit Biases: LLMs pre-trained on large-scale web data absorb latent stereotypes, often surfacing indirect harmful outputs not readily captured by keyword filters (e.g., gendered professional assumptions).
2. Implicit Offensiveness: Context-dependent expressions such as “he’s such a Karen” exemplify harmful language without overt slurs or profanity. Models optimized for explicit toxicity detection underperform on these implicit modalities.
3. Jailbreak Prompts: Adversarially constructed inputs, e.g., “Do anything now…,” can subvert static policy filters, revealing brittleness in LLM guardrails.

Existing benchmarks predominantly rely on binary Safe/Unsafe labels, insufficient for precision signaling and for capturing the distribution of sociocultural interpretations. GuardEval was constructed to comprehensively span nuanced harms by (a) providing granular subcategory labels, (b) integrating adversarial samples, and (c) preserving diverse human annotation perspectives.

2. Dataset Derivation, Annotation, and Taxonomy

GuardEval synthesizes thirteen public safety and ethics datasets into a coherent, richly structured corpus:

• Scope: 166,488 prompt instances and 64,484 response instances, each with annotation across 106 mutually exclusive subcategories under 23 top-level categories. The derivation of these categories integrates harm taxonomies from Weidinger et al. (2021), Shelby et al. (2023), and the Llama Guard policy schema.
• Category Taxonomy: Top-level classes include Hate/Identity Hate, Profanity, Gender Bias, Racial Bias (subdivided into Anti-Black, Anti-Asian, Anti-Hispanic, Other Race/Ethnicity, Implicit Stereotype), Offensive Language, Emotional Toxicity, Sexual (Adult/Minor), Violence, Criminal Planning, Privacy Leakage, Medical/Legal Advice, Harassment, and Jailbreak.
• Data Sources: Integrated datasets—Nemo-Safety, BeaverTails, ToxicChat, CoCoNot, WildGuard, Social Chemistry, ProSocial, PRISM, MetaHate, HateBase, MedSafetyBench, WildJailBreak, and UltraSafety—were selected for orthogonal coverage (prompt/response, adversarial, and community-annotated examples).
• Annotation Policy: All original fine-grained labels from the source datasets are preserved to avoid relabeling bias and to maintain dataset-specific normative judgments.
• Balancing and Quality: Per-source quotas and weight-based sampling regularize class distributions, giving greater prevalence to rare subcategories. A held-out test set of 6,700 prompt-only examples labeled as Safe/Unsafe was curated to avoid overfitting to extreme or dominant categories.

Table 1: Summary of GuardEval’s Data Composition

Split	Prompts	Responses	Label Schema
Training	~150K	~58K	106 subcategories
Validation	~10K	~3K	106 subcategories
Test	~6.7K	—	Safe/Unsafe

3. Evaluation Protocols and Performance Metrics

GuardEval operationalizes moderation as a binary classification problem (Safe vs. Unsafe), while retaining subcategory granularity for diagnostics. Metrics are computed as follows:

• For each class $c \in \{0,1\}$:

$$\text{Precision}_c = \frac{TP_c}{TP_c + FP_c}, \quad \text{Recall}_c = \frac{TP_c}{TP_c + FN_c}, \quad F1_c = 2\frac{\text{Precision}_c \times \text{Recall}_c}{\text{Precision}_c + \text{Recall}_c}$$

• Macro F1 (primary metric): $F1_{\mathrm{macro}} = \frac{1}{2}(F1_0 + F1_1)$.
• Micro-averaged F1 is available but less emphasized due to balanced test splits.

Performance is reported for both prompt-level and response-level classification, as well as stratified by category clusters (Bias, Overt Safety, Adversarial).

4. Model Fine-Tuning Methodology: GemmaGuard (GGuard)

GGuard is derived using QLoRA and PEFT LoRA adapters on Gemma3-12B:

• Base Model and Configuration:
  • Gemma3-12B, 4-bit quantization (bnb-4bit)
  • LoRA rank $r=16$, $\alpha=16$, dropout=0
  • Maximum sequence length: 4,096 tokens
  • Optimizer: AdamW (8-bit), initial learning rate $2\times10^{-5}$
  • Effective batch size: 64 (per-device micro-batch = 2, gradient accumulation = 32)
  • Linear learning-rate decay, 10 warm-up steps, 2,500 total training steps (≈1 epoch)
• Training Data: Unified GuardEval training splits and structured Alpaca-style safety prompts.
• Regularization: Implicitly enforced via weighted sampling; no additional data augmentation.

5. Experimental Results and Comparative Analysis

5.1 Prompt and Response Classification

• Prompt-level Macro F1: GGuard = 0.832; OpenAI Moderator = 0.641; Llama Guard3 = 0.614
• Response-level Macro F1: GGuard = 0.794; OpenAI = 0.571; Llama Guard3 = 0.556

5.2 Category-Specific Performance

Average macro F1 by category group:

Category Group	GGuard	OpenAI Mod.	Llama Guard3
Bias	0.78	0.62	0.59
Overt Safety	0.86	0.72	0.69
Adversarial Cases	0.80	0.55	0.50

GGuard exhibits marked improvements across implicit bias, overt harm, and adversarial prompt settings, indicating that multi-perspective, balanced datasets enhance both sensitivity and robustness.

5.3 Failure Analysis

Persistent errors are concentrated on highly figurative language (“metaphorical violence”) and multi-label border cases (e.g., simultaneously misogynistic and adversarial prompts). Annotator entropy is elevated in these cases, reflecting genuine social ambiguity and suggesting a role for human-in-the-loop adjudication.

6. Significance, Limitations, and Future Directions

GuardEval’s central innovation is the preservation of diverse human perspectives and fine-grained harm typologies. The multi-source, multi-perspective approach enables LLM moderators to distinguish not just overt toxicity but subtle, context-dependent harms, and to generalize to adversarial and borderline instances.

Limitations

• Modality: Although Gemma3 is multimodal, current GuardEval evaluations are text-only. Preliminary results on UnsafeBench suggest possible generalization to images, but comprehensive multimodal benchmarks are absent.
• Language: English-only coverage restricts applicability to global moderation contexts.
• Over-Censorship: GGuard’s improved recall of subtle harms increases the risk of false positives, particularly with historical or quoted content.

Recommendations

• Incorporate participatory annotation from affected communities to better reflect domain-specific norms.
• Introduce tiered severity bands (e.g., “Needs Caution” vs. “Block”) for policy gradient enforcement.
• Pair safe/unsafe continuations for each prompt to probe response consistency.
• Extend benchmarks to multimodal input for stress-testing vision–language moderators.

GuardEval, coupled with GGuard, fills critical gaps in the LLM moderation landscape by providing human-centered, fine-grained, and adversarially robust benchmarks, and demonstrates significant advances in fairness, interpretability, and robustness compared to extant moderation models (Machlovi et al., 22 Dec 2025).