CHILLGuard: Chinese LLM Safety Guardrail

• CHILLGuard is a Chinese-native safety guardrail that categorizes content using a 5-macro, 31-micro taxonomy tailored for Chinese regulatory and cultural nuances.
• It employs a multi-stage data pipeline and a generator–classifier collaborative framework with Model-aware Direct Preference Optimization to refine moderation performance.
• Evaluations on CHILLGuardTest show an F1 improvement up to 89.77, highlighting its effectiveness in detecting and mitigating unsafe content compared to baselines.

Searching arXiv for CHILLGuard and closely related papers to ground the article. CHILLGuard is a fine-grained, Chinese-native content safety guardrail for LLMs, designed for prompt-level and response-level moderation in Chinese deployment scenarios where regulatory priorities, cultural context, linguistic nuance, and implicit evasion differ from English-centric settings. It supports binary safe/unsafe detection together with a taxonomy of 5 macro categories and 31 micro categories, is trained on CHILLGuardTrain with 405,007 samples and evaluated on CHILLGuardTest with 51,745 samples, and is learned under a generator–classifier collaborative framework via Model-aware Direct Preference Optimization (Yu et al., 13 Jun 2026). In the supplied literature, the same label also appears in unrelated cryogenic contexts, so scope specification is necessary when the term is used bibliographically (Feenstra et al., 10 Sep 2025, Lancaster et al., 2020).

1. Taxonomic definition and problem setting

CHILLGuard was introduced to address a specific gap in LLM safety guardrails: existing systems perform well in English or multilingual settings but lack adaptation to Chinese-specific regulatory policies, cultural context, linguistic nuances, and fine-grained risk classification requirements. The system is explicitly motivated by the observation that Chinese online discourse often uses implicit expressions such as homophones, euphemisms, idioms, historical allusions, sarcasm, “quote-then-deny” rhetoric, pinyin abbreviations, and look-alike characters, all of which can evade literal filters (Yu et al., 13 Jun 2026).

The taxonomy consists of 5 macro categories and 31 micro categories. Macro A, “Violations of Core Socialist Values,” covers A1–A8, including inciting subversion of state power, endangering national security or the national image, inciting separatism, promoting terrorism or extremism, promoting ethnic hatred, promoting violence, obscenity, or pornography, disseminating false and harmful information, and other content prohibited by laws and regulations. Macro B, “Discriminatory Content,” spans B1–B9 and includes ethnic, religious, national, regional, gender, age, occupational, and health-based discrimination, together with a residual category for other forms of discrimination. Macro C, “Commercial Violations and Non-compliance,” spans C1–C5 and includes infringing intellectual property, violating business ethics, disclosing commercial secrets, using algorithm, data, or platform advantages for monopoly or unfair competition, and other illegal or non-compliant commercial activities. Macro D, “Infringement of Legitimate Rights and Interests,” spans D1–D7 and includes endangering physical or mental health, infringing portrait rights, reputation rights, honor rights, privacy rights, personal information rights, and other legitimate rights and interests. Macro E, “Failure to Meet Safety Demands of Specific Services,” includes E1 inaccurate content that severely contradicts scientific common sense or mainstream cognition and E2 unreliable content that fails to assist users (Yu et al., 13 Jun 2026).

Macro	Micro range	Focus
A	A1–A8	Core socialist values, national security, social order
B	B1–B9	Discriminatory content
C	C1–C5	Commercial violations and non-compliance
D	D1–D7	Legitimate rights and interests
E	E1–E2	Service safety and reliability

A central conceptual point is that CHILLGuard is not restricted to coarse harmful-versus-harmless classification. It is designed for fine-grained moderation and category-specific downstream action. Macro E is especially notable because it extends the safety frame beyond illicit or abusive content to inaccurate or unusable outputs in high-stakes contexts. This suggests a broader notion of “guardrail” than simple refusal logic.

2. Data construction pipeline and dataset composition

CHILLGuard addresses the scarcity of high-quality Chinese safety data through a scalable multi-stage data construction pipeline composed of retrieval-augmented generation, prompt-engineering-based rewriting for implicit harmful sample generation, and multi-model voting-based label calibration (Yu et al., 13 Jun 2026).

The retrieval-augmented generation stage uses sources including Quora, X, and Weibo. About 480,000 real-world texts were crawled with approximately 20 seed keywords per micro-category, expanded by Gemini 3.1 Pro to about 80 per micro-category, yielding 2,480 keywords in total. The corpus is encoded with bge-m3 embeddings and stored in a vector database. Queries are constructed using macro and micro labels plus random keywords; the system retrieves Top-100 items and uniformly samples 5. A prompt template instructs minimal edits to preserve intent while converting material into natural user prompts. To avoid over-refusal during harmful prompt generation, the pipeline uses Dolphin-Mistral-24B-Venice-Edition. This stage outputs 59,520 RAG-generated samples (Yu et al., 13 Jun 2026).

The prompt-engineering rewriting stage targets implicit harms characteristic of Chinese discourse. Its techniques include symbolization and morpheme transformation, cultural mapping and allusions, rhetorical irony, semantic nesting under “grand narratives,” logical induction with presupposed premises, boundary probing, fragmented process expressions, conditional assumptions, third-party relay, downplaying or joking tone, pseudoscience structural imitation, concept mismatches, hollow “authoritative” phrasing, and correct-but-unusable content. Starting from 46,742 real-world prompts, the process produces 109,312 rewritten samples. A uniform sample of 3,697 original prompts, including 3,100 benign prompts, is retained in the data, while the remainder serve as rewrite seeds (Yu et al., 13 Jun 2026).

Label calibration is performed with a jury of Qwen3-30B-Instruct, GLM-4.7-30B-Flash, InternVL3.5-38B-Instruct, and Yi-1.5-34B-Chat. Majority vote determines the binary safe/unsafe label, with DeepSeek-V3.2-685B serving as the tie-breaking adjudicator. Fine-grained labels across the 31 micro categories are assigned by DeepSeek-V3.2-685B. Post-processing includes deduplication, translation unification, length filters, format normalization, and strict split hygiene (Yu et al., 13 Jun 2026).

The resulting datasets are large and heterogeneous. CHILLGuardTrain contains 405,007 samples: PolyGuard_CN 213,315, OpenGuardrails CN 70,908, RAG 40,920, PE-Rewritten 77,351, and Original Real-world 2,513. Its safe ratio is 67.43% and unsafe ratio is 32.57%. CHILLGuardTest contains 51,745 samples: RAG 18,600, PE-Rewritten 31,961, and Original Real-world 1,184, with a safe ratio of 51.58% and unsafe ratio of 48.42%. Stratified sampling ensures balanced distribution across the 31 micro categories, and there is strictly no overlap between train and test sets (Yu et al., 13 Jun 2026).

A methodological limitation is explicitly acknowledged: inter-annotator agreement is not reported. The paper notes that, if measured, a common statistic is Cohen’s kappa,

$\kappa = \frac{p_o - p_e}{1 - p_e},$

where $p_o$ is observed agreement and $p_e$ is chance agreement (Yu et al., 13 Jun 2026).

3. Generator–classifier collaboration and Model-aware DPO

The model architecture follows a generator–classifier collaborative framework in which a rewritten generator $G$ produces increasingly hard adversarial prompts and a guardrail classifier $C$ learns from the combined seed and generated data (Yu et al., 13 Jun 2026).

Training proceeds iteratively. In Iteration 0, the classifier is trained on CHILLGuardTrain via supervised fine-tuning, producing $C^{(0)}$. In Iteration 1, $G^{(0)}$ rewrites each seed prompt $k=4$ times to form $D_{\text{gen}}^{(1)}$; this is merged with the seed data into $D_{\text{train}}^{(1)}$, and another supervised fine-tuning stage yields $p_o$0. In Iteration 2, $p_o$1 is labeled by $p_o$2 as safe or unsafe, each sample is scored for quality $p_o$3 by $p_o$4, difficulty levels $p_o$5–$p_o$6 are assigned, preference pairs are constructed, Model-aware Direct Preference Optimization is run to obtain $p_o$7, and the resulting generated data are merged again for a final supervised fine-tuning stage that produces $p_o$8 (Yu et al., 13 Jun 2026).

The classifier objective is multi-class cross-entropy over 31 categories, and optionally a binary head:

$p_o$9

During evaluation, unsafe is the positive class, and the classifier outputs fine-grained category labels for unsafe cases (Yu et al., 13 Jun 2026).

MDPO modifies DPO through model-aware batch adaptation. The implicit reward is

$p_e$0

For batch sample $p_e$1, the instance reward gap is

$p_e$2

The method introduces a responsiveness factor $p_e$3, uses a dynamic KL coefficient

$p_e$4

and updates the global mean according to

$p_e$5

Difficulty-aware pair construction prioritizes $p_e$6 and excludes $p_e$7 to avoid noisy signals (Yu et al., 13 Jun 2026).

The generator backbone used for MDPO fine-tuning is Dolphin3.0-Llama3.1-8B, while Dolphin-Mistral-24B-Venice-Edition is used in the data pipeline for RAG rewriting. The classifier backbones are Qwen3 models at 1.7B, 4B, and 8B parameters. Generator MDPO uses AdamW, 1 epoch, peak learning rate $p_e$8, cosine schedule, warmup 0.1, $p_e$9, $G$0, $G$1, and batch size 16. Generation uses vLLM, $G$2 rewrites per prompt, temperature 0.7, and max tokens 2048. Classifier supervised fine-tuning uses AdamW, peak learning rate $G$3, cosine schedule, warmup 0.1, 1 epoch, batch size 16, and max sequence length 4096. All text is unified to Chinese via a translation pipeline, and Chinese-native tokenization is handled by the Qwen3 tokenizer (Yu et al., 13 Jun 2026).

4. Evaluation, ablations, and observed failure modes

The evaluation protocol considers Precision, Recall, F1, and Accuracy, with F1 as the primary metric and unsafe as the positive class:

$G$4

On CHILLGuardTest, CHILLGuard-8B achieves an overall F1 of 89.77, representing a 15.92% improvement over Qwen3Guard-8B-Strict at 77.44. CHILLGuard-4B reaches 89.43, and CHILLGuard-1.7B reaches 82.72 (Yu et al., 13 Jun 2026).

Model	Overall F1	Note
CHILLGuard-8B	89.77	+15.92% over Qwen3Guard-8B-Strict
CHILLGuard-4B	89.43	Outperforms similarly sized baselines
CHILLGuard-1.7B	82.72	Outperforms similarly sized baselines

The macro-level F1 scores for CHILLGuard-8B are A 90.70, B 90.84, C 82.36, D 92.30, and E 90.99. On broader Chinese prompt datasets, CHILLGuard-8B obtains an average F1 of 77.05 across PolyGuardPrompts, WildGuardTest, ChineseSafe, DoNotAnswer, SafetyPrompts, and CHILLGuardTest. On response datasets, it achieves an average F1 of 79.16 across BeaverTails, PolyGuard responses, and RTP_LX. Its overall average F1 across prompts and responses is 77.75 (Yu et al., 13 Jun 2026).

The ablation results attribute gains to both collaborative training and implicit sample generation. Iteration 0, trained only on seed data, is improved by Iteration 1 with generated data; Iteration 2 with standard DPO improves further; and Iteration 2 with MDPO yields the best F1. For CHILLGuard-8B, the final gain cited is 88.85 to 89.77. Removing PE rewriting causes a significant drop across scales; at 8B, overall F1 declines from 89.77 with PE to 85.30 without PE, indicating that implicit harmful sample generation is critical (Yu et al., 13 Jun 2026).

Error analysis identifies several characteristic baseline failures: severe category imbalance, especially for Macro E and some B categories, missing implicit harms expressed through sarcasm, cultural allusions, and homophones, and over-flagging benign content. CHILLGuard reduces these failure modes through implicit rewrites, MDPO focus on hard pairs, and the fine-grained taxonomy, but the paper notes that it can still miss novel obfuscations or borderline cases requiring policy updates (Yu et al., 13 Jun 2026).

5. Deployment patterns, governance, and limitations

CHILLGuard is intended for several deployment patterns: pre-generation steering, in which user prompts are screened and safer formulations are suggested; real-time moderation, in which prompts or outputs are classified and unsafe content is blocked or transformed with category-specific guidance; and post-generation auditing, in which conversation logs are reviewed for compliance, batch scoring, and retraining-data curation (Yu et al., 13 Jun 2026).

Thresholding is configurable. Because unsafe is the positive class, deployment can tune decision thresholds to reduce false positives or false negatives according to risk appetite. The paper recommends treating Macro A with higher-priority thresholds. For Macro E, preferred responses include safe rewrites or deferrals to verified sources. Inputs that mix languages should be pre-normalized to Chinese, as in the training pipeline, or handled with multilingual tokenization. Context-aware screening is also necessary, since implicit harms may depend on preceding turns (Yu et al., 13 Jun 2026).

The system is offered in multiple parameter scales: 1.7B, 4B, and 8B. Smaller variants are described as suitable for edge or high-throughput moderation, while 8B yields the best accuracy. Throughput depends on the deployment stack and batching strategy; official latency numbers are not reported (Yu et al., 13 Jun 2026).

The governance frame is explicitly aligned with Chinese regulatory requirements and cultural context. The taxonomy and data construction prioritize national security, unity, rights protection, commercial compliance, and implicit Chinese expression patterns. Anti-discrimination granularity across B1–B9 is intended to support fair moderation across dialects, regions, occupations, and health status. Datasets were screened to remove personally identifiable information, and multi-model labeling together with stratified sampling is presented as a bias-mitigation measure. Planned release is under CC BY-NC 4.0 for non-commercial research use, with code, models, data, model cards, and training and evaluation scripts to be released through the project repository (Yu et al., 13 Jun 2026).

Several limitations remain explicit. Specialized industry categories may need expansion; robustness against emerging adversarial tactics requires continual data refresh and model iteration; and cross-lingual generalization remains future work. These limitations are consistent with the system’s design emphasis on Chinese-native moderation rather than universal multilingual transfer (Yu et al., 13 Jun 2026).

6. Term reuse and bibliographic disambiguation

Although CHILLGuard is most directly established as the Chinese LLM safety guardrail introduced in 2026, the supplied literature also uses the label in unrelated cryogenic contexts. One such use appears in connection with a cryogenic geometric anti-spring isolation system for closed-cycle cryostats. There, a cryogenic GAS implementation is presented “as a CHILLGuard solution,” using six radially arranged titanium blade springs with in-situ magnetic tuning, achieving a vertical resonance frequency of 185 mHz at 7 K and an order-of-magnitude reduction of vertical vibration at approximately 1 Hz (Feenstra et al., 10 Sep 2025).

A second unrelated use appears in a technical overview labeling cryogenic electrostatic guiding of the methylidyne radical CH as CHILLGuard. In that context, a cryogenic buffer-gas cooled CH beam is guided by a 200 mm hexapole with a 5 mm bore at alternating $G$5, with guided fluxes up to $G$6 molecules $G$7 per pulse, total guided numbers of approximately $G$8–$G$9 molecules per pulse, and guided transverse spreads corresponding to approximately 0.1 K near $C$0 kV (Lancaster et al., 2020).

This term reuse suggests that “CHILLGuard” is not intrinsically univocal across recent research-adjacent discourse. In citation, indexing, and retrieval settings, explicit disambiguation by arXiv identifier is therefore preferable. Within the present literature set, however, the standalone proper-name usage “CHILLGuard” denotes the Chinese-native LLM safety guardrail with a 5-macro, 31-micro taxonomy, scalable data construction, and Model-aware Direct Preference Optimization as its defining technical features (Yu et al., 13 Jun 2026).