Llama-3.1 Nemotron Safety Guard Multilingual 8B

• Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 is a multilingual content safety guard that uses PEFT LoRA fine-tuning on Llama-3.1-8B-Instruct to moderate prompts and responses in nine languages.
• It outputs a structured JSON with binary safe/unsafe judgments and multi-label safety categories, leveraging a culturally adapted training dataset for robust performance.
• The model achieves state-of-the-art harmful-F1 performance improvements, closing safety gaps in non-English scenarios through synthetic data augmentation and multilingual evaluation.

Searching arXiv for the model paper and closely related multilingual safety work. arxiv_search.query {"search_query":"ti:\"CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications\" OR all:\"Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1\"","max_results":5} arxiv_search.query {"search_query":"ti:\"LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety\"","max_results":5} arxiv_search.query {"search_query":"ti:\"Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture\" OR ti:\"MPO: Multilingual Safety Alignment via Reward Gap Optimization\"","max_results":10} Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 is a multilingual content safety guard built on top of the Llama-3.1-8B-Instruct backbone and specialized through PEFT LoRA fine-tuning to detect and categorize unsafe content across nine languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, Chinese, and English. It is designed to moderate both user prompts and agent responses, and to emit structured JSON containing binary safe/unsafe judgments together with a multi-label list of violated safety categories. The model is trained on Nemotron-Content-Safety-Dataset-Multilingual-v1, a culturally aligned multilingual expansion of Nemotron-Content-Safety-Dataset-V2 produced by the CultureGuard pipeline, and the reported final configuration achieves state-of-the-art performance on several multilingual content safety benchmarks while substantially improving over earlier English-only or monolingual guards in non-English settings (Joshi et al., 3 Aug 2025).

1. Model identity and intended function

Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 is a compact safety guard whose primary function is content moderation rather than open-ended generation. Its role is to determine whether an input prompt, or a prompt-response pair, is safe or unsafe and to assign one or more policy-relevant violation categories. In the reported formulation, it is described as a compact, commercially friendly guard model, and its exact designation is explicitly given as Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 (Joshi et al., 3 Aug 2025).

The model targets a specific failure mode of multilingual LLM deployment: general LLMs often exhibit markedly weaker safety behavior outside English. The reported benchmark observations motivating this guard include multilingual safety score drops relative to English for several open models. Examples include Llama-3.1-Nemotron-Nano-4B-v1.1 with 97.07 in English versus 67.27 in multilingual evaluation without English, Llama-3.2-3B-Instruct with 91.50 versus 84.77, and Qwen3-8B with 94.52 versus 88.93. Small models are reported as most affected, with the lowest safety often occurring in Hindi and Japanese, while Gemma-2 is noted as unusually close to English–multilingual parity (Joshi et al., 3 Aug 2025).

Relative to earlier English-only safety guards such as Llama-Nemotron-Safety-Guard-V2, the multilingual model is trained on a culturally aligned multilingual dataset rather than on translated English-only supervision. This design choice is intended to close performance gaps across languages by aligning the moderation behavior itself, rather than merely extending surface-language coverage.

2. Safety taxonomy and output interface

At inference time, the guard is prompted to produce a structured JSON object with three components: “User Safety”, “Response Safety”, and “Safety Categories.” The first two are binary safe/unsafe judgments; the third is a multi-label list of violated categories. This JSON interface is the central operational contract of the model and is intended to support deployment in moderation pipelines that require machine-readable outputs rather than free-form explanations (Joshi et al., 3 Aug 2025).

The labeling scheme is grounded in the Nemotron-Content-Safety-Dataset-V2 taxonomy. In the reported description, that taxonomy organizes safety risks into 12 top-level hazard categories with 9 fine-grained subcategories, excluding “safe” and “needs caution” from the hazard count. The inference prompt further expands this into a practical label inventory S1–S23, including examples such as Violence, Sexual, Criminal Planning/Confessions, Suicide and Self-Harm, Hate/Identity Hate, Harassment, Threat, Profanity, Fraud/Deception, Malware, Political/Misinformation/Conspiracy, Unauthorized Advice, and Illegal Activity (Joshi et al., 3 Aug 2025).

A common misconception is to treat the model as a conventional classifier head attached to a backbone. The reported implementation is different: it is trained as an instruction-tuned generator that emits JSON-formatted labels, and the outputs are subsequently parsed for evaluation. Loss details are not disclosed. The paper notes that instruction tuning typically optimizes token-level cross-entropy for generative outputs, whereas multi-label BCE is common for classifier heads, but it does not specify the use of a classifier head or BCE in this model. Likewise, class imbalance is not described as being handled by explicit reweighting; the reported mitigation comes from synthetic expansion and multilingual/cultural adaptation of the training data rather than loss-level weighting (Joshi et al., 3 Aug 2025).

3. CultureGuard data pipeline and multilingual supervision

The training corpus, Nemotron-Content-Safety-Dataset-Multilingual-v1, is produced by the CultureGuard pipeline, which consists of four explicitly named stages: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline starts from the English Nemotron-Content-Safety-Dataset-V2 and converts it into a multilingual, culturally aligned safety dataset spanning nine languages (Joshi et al., 3 Aug 2025).

In the first stage, cultural data segregation separates samples into “culturally influenced” and “generic” subsets using an LLM classifier. GPT-4o validation is reported at 95.32% test accuracy for this segregation step. In the second stage, cultural data adaptation uses Mixtral-8x22B to localize culturally influenced English samples into target-culture English while preserving the original intent and safety level. The reported example replaces a culture-specific ethnic reference while retaining harmful intent. Adaptation consistency is checked by a jury of LLMs—Llama-Nemotron-Safety-Guard-V2, PolyGuard-Qwen, Llama-3.1-Nemotron-70B-Instruct, and Qwen3-235B-A22B—and GPT-4o judges adaptation quality at an average 3.74/5 across languages (Joshi et al., 3 Aug 2025).

The third stage uses Google Cloud translation, chosen empirically for accuracy and non-refusal on harmful content, to scale the adapted English data into eight non-English languages. The fourth stage, quality filtering, combines a cross-lingual safety consistency filter with FAITH-based translation quality filtering. The cross-lingual safety consistency filter back-translates translated text into English and removes samples whose safety label diverges from the original. The FAITH filter evaluates Fluency, Accuracy, Idiomaticity, Terminology, and Handling of Format, using a low threshold of 3.5/5 to remove poor translations while minimizing data loss (Joshi et al., 3 Aug 2025).

The resulting dataset contains 386,661 samples in 9 languages. It also includes a multilingual jailbreak subset, CultureGuard-JB, built from 10k English pairs expanded via translation. The reported JB test split contains 8,883 multilingual samples, while a standard non-JB multilingual test split contains 17,676 multilingual samples (Joshi et al., 3 Aug 2025).

4. Fine-tuning configuration and model construction

The model is obtained by LoRA fine-tuning on top of Llama-3.1-8B-Instruct. The reported setup uses PEFT LoRA adapters with rank $r=8$ and alpha $=32$, trained for 5 epochs at a constant learning rate of $1\times10^{-5}$, with per-GPU batch size 4, distributed across 8 A100 GPUs under PyTorch FSDP. The LoRA update is written as $W = W_0 + BA$, with $A \in \mathbb{R}^{d \times r}$and$B \in \mathbb{R}^{r \times d}$ (Joshi et al., 3 Aug 2025).

Fine-tuning is conducted as instruction tuning rather than discriminative classifier training. Inputs are wrapped in the Llama 3.1 prompt format, and the model is trained to return a JSON object containing “User Safety,” “Response Safety,” and “Safety Categories.” This makes the guard interoperable with text-native LLM serving stacks while retaining a structured moderation interface. The paper does not disclose dropout, target layers, optimizer choice, context length, or weight decay (Joshi et al., 3 Aug 2025).

The final model configuration reported in the evaluation is “CultureGuard + JB SDG + QF.” Here, the ablation structure matters. Direct multilingual expansion without cultural adaptation is denoted “CultureGuard (vanilla)”; cultural adaptation then improves over direct translation; and further gains arise from adding JB SDG and QF. This indicates that the final guard is not merely a translated English safety model, but the result of layered data curation and augmentation.

The release plan states that both the dataset and the model will be publicly released under a commercial-permissive license. The work also points to reproducibility support through training recipes in the Llama Cookbook and prompt templates in the Appendix and model card (Joshi et al., 3 Aug 2025).

5. Evaluation protocol and reported performance

Evaluation uses harmful-F1 as the primary metric, with macro-averages computed by averaging language-wise scores. The paper does not report per-language threshold calibration, AUROC, AUPRC, or threshold selection procedures. Benchmarks cover internal test sets—CultureGuard and CultureGuard-JB prompts and responses—as well as public datasets including PolyGuardPrompts, RTP-LX, MultiJail, and XSafety (Joshi et al., 3 Aug 2025).

The final model, Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1, achieves an average harmful-F1 of 82.37 across benchmarks. Reported baseline comparisons include a 30.2% improvement over the English-only Llama-Nemotron-Safety-Guard-V2 (63.26 → 82.37) and a small but consistent gain over PolyGuard-Qwen, summarized as approximately 1.03 points overall mean in Table 1, while the abstract reports approximately 1.3 (Joshi et al., 3 Aug 2025).

Benchmark	Input type	harmful-F1
CultureGuard test set	Prompts	85.15
CultureGuard test set	Responses	85.48
CultureGuard-JB test set	Prompts	91.77
CultureGuard-JB test set	Responses	94.35
PolyGuardPrompts	Prompts	79.25
PolyGuardPrompts	Responses	72.89
RTP-LX	Prompts	91.49
MultiJail	Prompts	95.36
XSafety	Prompts	66.97

On the CultureGuard multilingual test set, prompt harmful-F1 is reported per language as en 86.50, ar 84.77, de 85.33, es 84.92, fr 85.46, hi 84.12, ja 85.44, th 84.07, zh 85.77. Response harmful-F1 is reported as en 86.36, ar 85.30, de 83.96, es 86.56, fr 86.12, hi 84.72, ja 86.20, th 84.95, zh 85.15. The spread is comparatively narrow across the supported languages, which is consistent with the paper’s claim of strong parity and robustness in non-English settings (Joshi et al., 3 Aug 2025).

The ablation trend is also explicit. CultureGuard (vanilla) averages 79.96; adding cultural adaptation raises this to 81.04; and JB SDG + QF yields the final 82.37. This indicates that cultural localization, jailbreak-oriented synthetic augmentation, and translation quality filtering each contribute measurable performance gains rather than serving as interchangeable preprocessing steps.

6. Broader research context, deployment implications, and limitations

The model sits within a broader line of multilingual safety research that distinguishes between three related problems: multilingual moderation, multilingual alignment of base policies, and adversarial evaluation under mixed-language conditions. The guard model directly addresses multilingual moderation, but adjacent work clarifies both its likely strengths and its remaining blind spots.

First, LASA identifies a semantic bottleneck in multilingual instruction-tuned LLMs and proposes Language-Agnostic Semantic Alignment by attaching a lightweight Safety Semantic Interpreter at the bottleneck layer. In the integration guidance, LASA is presented as complementary to Nemotron-style multilingual safety guards: output-layer safety classifiers operate in language-dominant layers, whereas a mid-layer semantic detector can add cross-lingual robustness, especially for low-resource languages. The paper explicitly frames this as a possible enhancement path for systems based on Llama-3.1-8B, including a multilingual Nemotron safety guard (Yang et al., 13 Apr 2026). This suggests that the current guard’s JSON-based moderation interface and LASA’s mid-layer safety signal are architecturally compatible rather than mutually exclusive.

Second, Multilingual Blending studies mixed-language query-response schemes in which malicious prompts are transformed into token-level mixtures across multiple languages and the model is instructed to reply in the same mixed format. That work reports sharply increased bypass rates under mixed-language conditions and recommends evaluation and defense strategies such as language identification, normalization, translation-to-canonical-policy checks, multilingual refusal scaffolds, and uncertainty-aware gating (Song et al., 2024). A plausible implication is that a multilingual guard trained on nine languages should not be interpreted as automatically robust to arbitrary code-switched or token-level blended attacks, because the guard paper does not report direct evaluation under that threat model.

Third, MPO approaches multilingual safety from a different angle: it aligns the base model’s target-language reward gap to a dominant-language safety anchor, using representation retention to preserve English capabilities. MPO is evaluated on base-policy alignment rather than on a JSON-emitting guard model, but it reports strong reductions in attack success rate for LLaMA-3.1-8B-Instruct while preserving utility (Zhao et al., 22 May 2025). This suggests an alternative training paradigm for multilingual safety that could, in principle, complement a dedicated guard by improving the safety of the underlying generative model itself.

Several limitations are explicitly noted for the guard and its data pipeline. Cultural adaptation can be superficial or may return unmodified text; back-translation errors can cause overly aggressive filtering; confidence scores, latency, throughput, memory footprint, and quantization details are not reported; and the paper does not provide category-level class distributions or explicit threshold calibration procedures (Joshi et al., 3 Aug 2025). Ethical considerations are also central: the dataset contains critically unsafe and offensive content for training purposes, and users are urged to exercise extreme caution. The model’s practical significance therefore lies less in any claim of universal multilingual safety than in a documented, data-centric attempt to standardize multilingual moderation behavior across nine languages while retaining an operationally simple JSON interface and a compact 8B deployment profile.