Libra-Guard: Chinese LLM Safety Filter

• Libra-Guard is a safeguard system for Chinese LLMs that assesses query-response pairs based on predefined safety rules.
• It employs a two-stage curriculum training pipeline with synthetic pretraining and real-world finetuning to enhance robustness.
• Empirical results show superior accuracy over comparable models, establishing a robust benchmark for Chinese content moderation.

Libra-Guard is a safeguard system specifically built for Chinese-language LLMs. Rather than generating user-facing answers, it takes a query-response pair as input and predicts whether the response is “Safe” or “Unsafe” under a set of safety rules. Introduced together with Libra-Test, the first benchmark specifically designed to evaluate the effectiveness of safeguard systems for Chinese content, Libra-Guard is trained with a two-stage curriculum pipeline—guard pretraining on synthetic samples followed by finetuning on high-quality real-world data—and achieves 86.79% accuracy on Libra-Test, outperforming Qwen2.5-14B-Instruct at 74.33% and ShieldLM-Qwen-14B-Chat at 65.69%, while nearing Claude-3.5-Sonnet and GPT-4o (Chen et al., 29 Jul 2025).

1. Functional Scope and Intended Role

Libra-Guard is framed as a safeguard system for Chinese-based LLMs rather than as a general-purpose assistant or response generator. Its operational objective is to judge whether an LLM response is unsafe under predefined safety rules. In deployment, the system consumes a query-response pair and outputs only the final label; during training, it can additionally produce a “critic,” namely an explanation of why the label is chosen. The omission of the critic at inference time is explicitly motivated by fast deployment in practical moderation settings (Chen et al., 29 Jul 2025).

The paper positions Libra-Guard as a response to a structural gap in the safety ecosystem. Existing guard models are described as English-centric, heavily dependent on manual annotation, and insufficiently effective at exploiting synthetic data. Libra-Guard is therefore designed to be more scalable and more suitable for Chinese content moderation by combining synthetic data, real-world data, and a curriculum-style training pipeline. In the paper’s practical framing, the target risk surface includes hate speech, crime, misinformation, discrimination, privacy violations, and related harms that are especially sensitive in Chinese-language deployment.

A common misconception would be to treat Libra-Guard as a model for directly rewriting or improving unsafe answers. The system described in the paper is narrower and more specialized: it is a safety filter whose primary function is adjudication. This distinction matters because the evaluation focuses on safeguard effectiveness rather than on general language ability.

2. Training Pipeline and Data Construction

The central methodological idea is a two-stage curriculum training pipeline: guard pretraining on large-scale synthetic samples, followed by guard finetuning on higher-quality real-world samples. The paper explicitly invokes curriculum learning: the model is first exposed to easier, broader, synthetic supervision and then refined on harder, more realistic examples. The stated purpose is to improve sample efficiency and robustness while reducing reliance on expensive human annotation (Chen et al., 29 Jul 2025).

In the guard pretraining stage, the authors synthesize large numbers of harmful Chinese queries. They follow the spirit of AART, but extend query generation by conditioning not only on harmful behavior, task type, and country context, but also on specific harmful events. The raw queries are then rewritten, paraphrased, red-teamed, and deduplicated at the semantic level. For each synthesized harmful query, responses are generated using Qwen-2.5-7B, Llama-3.1-8B, and Gemma-2-9B, in both base and instruct variants, to ensure coverage of both safe and unsafe completions. These query-response pairs are then automatically labeled for safety using Qwen-2.5-32B-Instruct and Gemma-2-27B-it according to predefined safety rules. Samples where both annotators agree are retained as “easy samples,” and safe and unsafe responses are balanced by sampling one of each per query. This stage produces about 240k synthetic pretraining instances.

In the guard finetuning stage, the model is trained on high-quality real-world data. Harmful queries are sampled from the Safety-Prompts dataset, with no overlap with the real-data portion of Libra-Test. Responses are generated in the same general way as in pretraining, but the annotation flow is more selective: weaker models first identify inconsistently labeled responses, which are treated as “hard samples,” and those are then relabeled by GPT-4o using the same safety rules. After balancing safe and unsafe examples, the resulting finetuning set contains about 18k instances. The paper presents this stage as the mechanism that sharpens the model on difficult, realistic cases.

The optimization setup is standard supervised training rather than a novel explicit objective. Libra-Guard is pretrained on the approximately 240k synthetic instances and then fully finetuned on the approximately 18k real harmful-query instances. Training uses the Adam optimizer with a linear decay schedule, batch size 384, and two epochs for pretraining and one epoch for finetuning. The model is initialized from aligned open-source LLMs. The prompt used during training instructs the model to act as a text safety expert and output in strict JSON format. Depending on configuration, the prompt can include safety rules and place the critic either before the label or after it; the best-performing configuration is “Rear Critic,” where the critic follows the label.

3. Libra-Test Benchmark and Safety Taxonomy

Libra-Test is introduced as the evaluation benchmark paired with Libra-Guard and is motivated by the claim that no dedicated benchmark existed for evaluating safeguard systems on Chinese content. Its design is organized around three principles: diversity, difficulty, and consistency. The benchmark is intended to measure the protective capacity of safety models rather than merely their general language ability (Chen et al., 29 Jul 2025).

For diversity, Libra-Test combines three sources: real Chinese harmful questions from Safety-Prompts paired with responses from various LLMs; synthetic data generated with the same query-generation methods used in training; and translated data, in which English safety benchmarks such as BeaverTails are translated into Chinese so that scenarios absent in native Chinese data are still covered. For difficulty, the benchmark deliberately retains hard examples: Qwen-2.5-32B-Instruct and Gemma-2-27B-it first label responses, samples on which they disagree are retained, and those cases are then manually annotated. For consistency, Libra-Test uses a unified set of safety rules, and each sample is independently annotated by three human annotators, then labeled by majority vote, and finally confirmed by a safety expert.

Source	Samples
Real	1,262
Synthetic	1,467
Translated	2,991

The final benchmark contains 5,720 samples, of which 1,864 are safe and 3,856 unsafe. The seven harm scenarios are Physical and Mental Health; Privacy and Property; Ethics and Morality; Bias and Discrimination; Illegal Activities and Crime; Hate Speech and Abuse; and Rumors and Misinformation. These categories define what counts as unsafe both in the benchmark and in the training labels.

The benchmark construction has methodological significance beyond simple dataset release. By mixing real, synthetic, and translated material while preserving disagreement-heavy examples, Libra-Test is meant to stress-test safeguard systems under heterogeneous data styles and annotation difficulty. This suggests that benchmark quality is treated as part of the safeguard problem itself, not merely as an external measurement convenience.

4. Empirical Results and Comparative Performance

Libra-Guard is evaluated on Libra-Test using Accuracy and F1 scores, specifically F1-Safe and F1-Unsafe. The strongest reported model, Libra-Guard-Qwen2.5-14B-Instruct, reaches 86.79% average accuracy, with 80.64% F1-Safe and 89.83% F1-Unsafe. This substantially exceeds Qwen2.5-14B-Instruct at 74.33% average accuracy and ShieldLM-Qwen-14B-Chat at 65.69%. It also outperforms older or smaller open-source instruction models such as Qwen-7B, Qwen-3B, Yi-1.5-9B-Chat, Llama-Guard3-8B, and ShieldGemma-9B (Chen et al., 29 Jul 2025).

System	Average accuracy
GPT-4o	91.05%
Claude-3.5-Sonnet	88.82%
Libra-Guard-Qwen2.5-14B-Instruct	86.79%
Qwen2.5-14B-Instruct	74.33%
ShieldLM-Qwen-14B-Chat	65.69%

The scale trend is also explicit. Libra-Guard consistently improves over the base model across sizes: the 0.5B, 1.5B, 3B, 7B, and 14B versions all perform well, and performance increases with scale. The 0.5B version already reaches 81.46%, while the 14B version reaches 86.79%. The paper interprets this as evidence that the training recipe is effective even at small scales but continues to benefit from larger backbones.

Comparison with closed-source models is presented as an important reference point rather than as a claim of parity. GPT-4o achieves 91.05% average accuracy and Claude-3.5-Sonnet 88.82%. On translated data, GPT-4o reaches 94.78% and Libra-Guard-Qwen2.5-14B-Instruct reaches 91.04%; on real data, Libra-Guard gets 85.97% compared with GPT-4o’s 88.59%; on synthetic data, Libra-Guard gets 83.37% compared with GPT-4o’s 89.78%. The gap indicates that frontier closed models still retain an edge, but Libra-Guard establishes a strong open-source baseline tailored for Chinese safety filtering.

5. Ablation Findings and Design Choices

The ablation study supports the core design decisions of the system. Scaling synthetic pretraining data improves accuracy from roughly 83.5% to 86.5%, and scaling finetuning data also improves performance. Pretraining helps more than training from scratch, especially at small data sizes. Most importantly, the curriculum strategy Pretrain → SFT outperforms both Pretrain + SFT mixed training and single-stage baselines, achieving 86.48% in the ablation setup (Chen et al., 29 Jul 2025).

The treatment of data difficulty is also consequential. An ablation shows that using “hard samples” in finetuning is better than using only easy samples, reinforcing the value of difficult real-world cases in the second stage. Likewise, the paper reports that combining annotator models matters: when Qwen and Gemma are used together and only agreed-upon labels are retained, performance improves over using either model alone. This suggests that stricter agreement-based filtering yields cleaner synthetic and finetuning data.

The critic ablation is one of the clearest architectural findings. Three setups are compared: No Critic, Front Critic, and Rear Critic. Rear Critic performs best, reaching 86.48% average accuracy, compared with 81.80% for No Critic and 82.34% for Front Critic. The result indicates that explanation-aware training helps, but that the position of the explanation relative to the label materially affects performance.

Another notable finding is that explicitly including safety rules in the prompt is not essential. Models trained with or without the rules perform similarly, and in one ablation omitting the rules slightly improves performance. The authors interpret this as evidence that Libra-Guard internalizes the safety principles rather than relying on prompt instructions at inference time. A plausible implication is that the training pipeline, rather than prompt verbosity, is the dominant source of safeguard behavior.

6. Governance Significance, Misconceptions, and Limitations

Libra-Guard and Libra-Test are presented as a coherent safety framework for Chinese LLMs: Libra-Guard supplies the safeguard model, and Libra-Test supplies the evaluation substrate. The broader implication drawn in the paper is that safety governance for Chinese LLMs should not simply import English-centric moderation methods; it requires language-specific benchmarks, safety rules, and training pipelines (Chen et al., 29 Jul 2025).

This framing addresses a recurrent misconception in LLM safety discourse: that moderation models are language-agnostic once they are sufficiently large. The work instead emphasizes Chinese-specific data construction, Chinese-specific annotation, translated coverage for missing scenarios, and a benchmark explicitly targeted at safeguard systems rather than at general instruction following. In that sense, Libra-Guard is not merely a Chinese reimplementation of existing English guard models; it is a pipeline optimized for the data, annotation, and deployment conditions identified in the paper.

The limitations are described more implicitly than polemically. Libra-Guard still trails frontier closed-source systems, the benchmark focuses on text rather than multimodal input, and the safety rules reflect one fixed policy framing rather than an explicitly plural policy space. The paper nevertheless treats the system as a tentative step toward safer and more reliable Chinese AI systems. It also points toward future directions such as multimodal safeguards, long-context safety, and stronger reasoning-aware moderation.

Taken together, Libra-Guard occupies a specific place in the safety stack: it is a scalable, open-source safeguard model for Chinese content moderation, trained through synthetic-to-real curriculum learning and evaluated on a benchmark constructed to stress diversity, difficulty, and consistency. Its significance lies less in proposing a new loss function than in integrating data synthesis, agreement-based filtering, hard-sample finetuning, explanation-aware training, and Chinese-specific evaluation into a single safeguard workflow.