CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment

Abstract: Malicious content generated from LLMs could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilingual settings, they lack adaptation to Chinese-specific regulatory policies, cultural context and linguistic nuances, failing to support fine-grained risk classification for diverse deployment needs. In this paper, we introduce a 5-macro, 31-micro category fine-grained risk taxonomy for Chinese scenarios, and build CHILLGuard: a dedicated Chinese LLM content safety guardrail. To address the critical scarcity of high-quality annotated Chinese safety data, we propose a scalable multi-stage data construction pipeline: we expand multi-source corpus via retrieval-augmented generation, generate implicit harmful samples through prompt engineering rewriting, and refine high-quality data via multi-model voting-based label calibration. Based on this, we build CHILLGuardTrain, a large-scale training set with 405,007 samples, and CHILLGuardTest, a rigorously curated annotated test set with 51,745 samples. We then train CHILLGuard on CHILLGuardTrain under a generator-classifier collaborative framework via Model-aware Direct Preference Optimization. Extensive experiments under multiple settings demonstrate the state-of-the-art performance of CHILLGuard, e.g., a 15.92% improvement of F1 score over Qwen3Guard-8B-Strict on our benchmark. We will release our resources at https://github.com/cswbyu/CHILLGuard.

• The paper introduces CHILLGuard, a safety framework that designs a 5 macro/31 micro-category Chinese safety taxonomy for LLM moderation.
• It employs a scalable dataset construction integrating retrieval-augmented generation, adversarial prompt rewriting, and multi-model label calibration.
• Experiments demonstrate that model-aware preference optimization via MDPO vastly improves robustness, achieving an F1 of 89.77 on a dedicated test set.

Fine-Grained Safety Guardrails for Chinese LLMs via Scalable Data Construction and Model-Aware Preference Alignment

Introduction and Motivation

The landscape of LLM safety for non-English geographies, particularly Chinese, presents regulatory, linguistic, and cultural complexities that render direct adaptation of English-centric guardrails ineffective. The paper "CHILLGuard: Towards Fine-Grained Chinese LLM Safety Guardrail with Scalable Data Construction and Model-aware Preference Alignment" (2606.15396) introduces a highly granular Chinese safety taxonomy (5 macro-categories, 31 micro-categories) and a dedicated Chinese moderation system—CHILLGuard—addressing the acute paucity of realistic, annotated Chinese safety data and tailored alignment techniques.

Dataset Construction Pipeline

A core contribution is the scalable dataset construction framework—integrating retrieval-augmented generation (RAG), prompt engineering (PE)-based adversarial rewriting, and multi-model label calibration—to create two high-quality datasets: CHILLGuardTrain (405k samples) and CHILLGuardTest (52k samples). This pipeline synthesizes authentic, implicitly harmful, and diverse queries intrinsic to real-world Chinese online discourse, with strict balancing across all categories.

Figure 1: Data pipeline unifying multi-source expansion, preprocessing, model-based label calibration, and final dataset curation for robust, category-diverse Chinese safety moderation.

Three data sources assure coverage:

• RAG-based expansion: Seeds expanded by Gemini 3.1 Pro, surface-level context preserved via targeted minimal rewrites by uncensored open models, informed by context embeddings (bge-m3).
• Authentic deployment prompts: Real enterprise traffic annotated by expert committees to reflect in-situ attack tactics.
• Prompt engineering rewriting: Systematic adversarial transformation exploiting typographic, phonetic, rhetorical, and contextual nuances of Chinese (homophones, allusions, sarcasm, boundary probing).

Unified preprocessing (translation, deduplication, normalization) ensures quality. Label calibration employs a panel of foundation LLMs via majority voting and fallback arbitration (DeepSeek-685B), removing annotation noise and enforcing fine-grained compliance.

Harm Taxonomy Design

Existing taxonomies are demonstrably insufficient for the regulatory and cultural idiosyncrasies of China. CHILLGuard defines the most comprehensive category structure to date for this domain:

• Macro A: State and regulatory security.
• Macro B: All axes of discrimination.
• Macro C: Commercial rights and compliance.
• Macro D: Individual legal rights (privacy, reputation, etc.).
• Macro E: Service fitness/quality violations.

Every sample is mapped at both macro and micro levels, enabling class-balanced detection and fine-grained enforcement.

Model-Aware Collaborative Training with MDPO

The moderation model employs a generator-classifier collaborative training loop, inspired by two-player game alignment frameworks (e.g., DuoGuard [deng2025duoguard]), but augmented with Model-aware Direct Preference Optimization (MDPO)—a significant departure from uniform-penalty DPO schemes. Here, the KL penalty on preference pairs is adaptively modulated according to instance-level hardness (reward gap outlier detection, sample-level masking), forcing the classifier to prioritize resolution of hard, adversarial cases and enabling robust discrimination even on highly implicit attacks.

Figure 2: Collaborative multi-iteration training where a rewritten generator and safety classifier iteratively harden each other using MDPO-based instance difficulty signals and adversarial prompt augmentation.

The process:

• Seed data is used to bootstrap the classifier.
• Generator produces hard-to-detect adversarial rewrites via PE; classifier is retrained on these until convergence.
• Difficulty-annotated, preference-paired data is continually generated, and MDPO ensures optimizer attention is focused on model-blind spot regions.
• Ablation studies confirm performance collapses when either multi-iteration co-training or MDPO’s dynamic penalties are ablated.

Empirical Evaluation

CHILLGuard exhibits markedly superior parameter efficiency and robustness. For example, at 8B scale, it delivers an F1 of 89.77 on CHILLGuardTest, exceeding Qwen3Guard-8B-Strict by 15.92 points. Even the 1.7B model outperforms 8B+ SFT baselines by large margins, demonstrating that robust adversarial data and targeted preference alignment sharply dominate naive scaling on this task.

Cross-category results show uniformity in CHILLGuard's risk coverage—there are no macro/micro-categories with F1 collapse, unlike all competitors. In particular, the system addresses the chronic failure modes of existing guardrails on discrimination (B), service safety (E), and implicit or boundary-pushing content. External Chinese benchmarks (PolyGuardPrompts, BeaverTails, etc.) confirm strong generalization.

Ablations on the PE mechanism, MDPO, and iterative training decisively prove that both the diversity and implicitness of adversarial data, as well as instance-specific penalty tuning, are critical to SOTA moderation—standard SFT or static DPO are insufficient.

Theoretical and Practical Implications

The results demonstrate that:

• Granular risk taxonomy and data curation are mandatory for compliance in high-regulation, high-context languages.
• Uniform penalty preference optimization is grossly suboptimal for classification-based safety guardrails; model-aware instance-adaptive alignment is necessary.
• Robust safety is not mere function of scale; data/programmatic adversarial coverage and fine-grained preference feedback are dominant.

On the practical side, CHILLGuard provides the first infrastructure-ready Chinese LLM moderation suite, with released datasets and alignment code as a public asset.

Limitations and Prospective Directions

While CHILLGuard establishes a robust baseline, its current taxonomy may require extension for specialized verticals, and adversarial robustness still relies on currently observable attack vectors—targeted adaptation to continuously evolving evasion tactics and cross-linguistic generalization remain future work.

Conclusion

CHILLGuard establishes a new standard for Chinese LLM safety moderation: combining a comprehensive taxonomy, scalable adversarial dataset construction, and a collaborative, model-aware alignment approach that achieves exceptional accuracy and robustness. Its paradigm—fine-grained risk taxonomies calibrated with model-aware preference optimization—prescribes the future direction for high-compliance, culturally-specific LLM safety infrastructure.