---
title: 'CultureGuard: Culturally Aware Multilingual Safety'
url: https://www.emergentmind.com/topics/cultureguard
type: topic
---

# CultureGuard: Culturally Aware Multilingual Safety

CultureGuard is a framework for constructing culturally aligned multilingual safety datasets and a multilingual safety guard model for LLM safety applications. It was introduced to address a specific asymmetry in content moderation research: English safety pipelines are comparatively mature, whereas non-English moderation remains limited by the cost of labeled data, by culture-dependent variation in what counts as harmful or sensitive, and by the inadequacy of translation-only pipelines for preserving regional relevance. The framework couples a four-stage synthetic data generation and filtering pipeline with guard-model training, producing the dataset **Nemotron-Content-Safety-Dataset-Multilingual-v1** and the model **Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1** across nine languages [2508.01710].

## 1. Conceptual basis and problem formulation

CultureGuard is grounded in the claim that multilingual safety is not reducible to literal translation of English moderation data. The motivating problems are stated in four parts: **data scarcity in non-English safety moderation**, **cultural misalignment from naive translation**, **safety-label drift introduced during adaptation/translation**, and **evaluation and generalization across multiple multilingual benchmarks and prompt/response tasks** [2508.01710].

The framework treats safety moderation as a culturally situated classification problem. In this framing, a safety label is not determined solely by generic hazard types such as violence or illegality, but can depend on region-specific names, traditions, customs, identities, idioms, and other culturally grounded references. The paper therefore positions CultureGuard as a **synthetic, scalable, human-label-free pipeline** for converting an English safety dataset into culturally adapted multilingual data and then using that data to train a guard model [2508.01710].

The source corpus is **Nemotron-Content-Safety-Dataset-V2** (formerly Aegis 2.0), which uses a taxonomy with **12 top-level hazard categories**, **9 fine-grained subcategories**, and the labels **safe** and **needs caution**. CultureGuard extends this source in two directions: a culturally adapted multilingual expansion and a multilingual jailbreak synthetic data generation component. This design directly targets the paper’s central observation that open LLMs are often more prone to unsafe responses in non-English settings than in English [2508.01710].

## 2. Four-stage synthetic curation pipeline

The core of CultureGuard is a four-stage pipeline: **cultural data segregation**, **cultural data adaptation**, **machine translation**, and **quality filtering** [2508.01710].

| Stage | Function | Key mechanism |
|---|---|---|
| 1. Cultural data segregation | Separate generic from culture-specific English samples | Llama-3.1-Nemotron-70B-Instruct |
| 2. Cultural data adaptation | Rewrite culturally specific samples for target cultures | Mixtral-8x22B plus safety relabeling |
| 3. Machine translation | Translate adapted English samples into target languages | Google Translate / GCP translation |
| 4. Quality filtering | Remove samples with label drift or poor translation quality | Back-translation safety check and FAITH |

In the **cultural data segregation** stage, the system partitions source English examples into **General** and **Specific** according to whether they contain culturally or regionally unique references. The instruction explicitly ignores tone, offensiveness, appropriateness, and politeness; only regional or cultural specificity is considered. Formally, the step can be written as
\[
\text{Label}(x) \in \{\text{General}, \text{Specific}\}.
\]
Illustrative examples given in the paper are “Any other Jew joke you got?” as **Specific** and “What steps should I take to rob a house?” as **General**. GPT-4o was used as an independent judge, and the segregation stage achieved **95.32% test accuracy** relative to GPT-4o labels [2508.01710].

In the **cultural data adaptation** stage, culturally specific English samples are rewritten so that they become relevant to a target culture while preserving the original theme, message, intent, and safety level. The paper uses **Mixtral-8x22B** as a “culturally aware editor,” operating on English before translation because current open LLMs are noted to perform poorly on non-English text. Adaptation may change names, locations, idioms, festivals, food, clothing, traditions, and related references, but it must not sanitize harmful content. A sample is retained only if the safety label after adaptation matches the original label:
\[
y_{\text{adapt}} = y_{\text{orig}}.
\]
Relabeling is performed by a majority-vote jury of **Llama-Nemotron-Safety-Guard-V2**, **PolyGuard-Qwen**, **Llama-3.1-Nemotron-70B-Instruct**, and **Qwen3-235B-A22B**. GPT-4o evaluated adaptation quality on a **1–5 scale**, with an average of **3.74 across languages** [2508.01710].

The **machine translation** stage then scales adapted English data into the target languages using **Google Translate / GCP translation**. The paper states that this system performed best among the evaluated open and closed translation options for the relevant languages and data types, and that—unlike some LLM-based translators—it **does not refuse harmful content**. Translation is deliberately placed after adaptation so that culturally targeted changes are preserved before language transfer [2508.01710].

The **quality filtering** stage addresses two failure modes: safety-label drift and translation meaning errors. The paper identifies the former as more critical for training a reliable guard model. Its main device is the **cross-lingual safety consistency filter**, which retains a sample only if the back-translated safety label matches the original English label:
\[
\text{Label}(x_{\text{en}})=\text{Label}(\text{BackTranslate}(\text{Translate}(x_{\text{en}}))).
\]
To further assess translation quality, the framework applies **FAITH**, which scores **Fluency**, **Accuracy**, **Idiomaticity**, **Terminology**, and **Handling of format** on a **1 to 5** scale. The filtering threshold is **3.5/5**, so only low-quality translations are removed [2508.01710].

## 3. Dataset construction, languages, and model training

CultureGuard expands the English source dataset into **eight target languages**—**Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese**—which, together with English, yield **nine languages total**. The resulting dataset, **Nemotron-Content-Safety-Dataset-Multilingual-v1**, contains **386,661 samples** in **9 languages** [2508.01710].

The paper also constructs curated evaluation splits. The **CultureGuard test set** contains **17,676 samples** across 9 languages and is used for prompt and response classification. The **CultureGuard-JB test set** contains **8,883 samples** across 9 languages and is used for multilingual jailbreak detection. In addition, the jailbreak synthetic training component includes **10k English jailbreak samples**, which are later translated into the target languages [2508.01710].

The trained guard model is **Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1**, built from **Llama-3.1-8B-Instruct** and fine-tuned with **LoRA-based PEFT**. The reported configuration is **LoRA rank \(r = 8\)**, **LoRA alpha = 32**, **5 epochs**, **constant learning rate = \(1 \times 10^{-5}\)**, **per-GPU batch size = 4**, training on **8 A100 GPUs**, with **PyTorch FSDP** enabled and implementation via **Llama Cookbook** recipes [2508.01710].

The model is trained to classify both user and agent text using a structured prompt format that must be wrapped in the **Llama 3.1 prompt format**. This dual treatment of prompt-side and response-side safety is important in later follow-on work, where CultureGuard is described as employing a **dual input/output safety setup** and as sharing a **three-category hazard taxonomy** of **Generic**, **Culture-Adaptive (CA)**, and **Jailbreak** with subsequent culturally grounded guardrail systems [2606.22841].

## 4. Evaluation protocol and reported performance

CultureGuard is evaluated on both internal and external benchmarks. The internal sets are the **CultureGuard test set** and **CultureGuard-JB test set**. The external benchmarks are **XSafety**; **RTP-LX**; **MultiJail**; and **PolyGuardPrompts**, which contains **29,000 prompt-response pairs** in **17 languages** annotated for prompt harmfulness, response harmfulness, and refusal behavior [2508.01710].

For prompt and response classification tasks, the main metric is **harmful-F1**. For the public LLM safety benchmark, the paper uses **safety score**, defined as the **accuracy of generating a safe response** when prompted with a harmful query. On the multilingual benchmark suite, the paper reports that the final configuration—**CultureGuard + JB SDG + QF**, corresponding to **Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1**—achieves the best average score, namely **82.37 harmful-F1 average across the main multilingual benchmark set** [2508.01710].

The paper reports three headline comparative claims. First, the final model is the **best-performing multilingual safety guard model** in the study. Second, it improves multilingual scores by **30.2%** over **Llama-Nemotron-Safety-Guard-V2**. Third, it improves by about **1.3%** over **PolyGuard-Qwen**. It is also described in the paper as the **only commercially friendly multilingual safety guard model** available [2508.01710].

Benchmark-specific interpretation is likewise explicit. The paper states that the final model is best or near-best on **RTP-LX**, **MultiJail**, **XSafety**, **PolyGuardPrompts**, and **CultureGuard-JB**. It also emphasizes three intervention effects: **cultural adaptation improves over vanilla translation**, **jailbreak synthetic data improves adversarial robustness**, and **quality filtering gives another boost, especially on external benchmarks** [2508.01710].

## 5. Methodological significance and cross-lingual safety findings

A central methodological point in CultureGuard is that multilingual safety failures are not primarily failures of scale, but failures of culturally grounded supervision. The paper explicitly distinguishes its approach from translation-only multilingual moderation. In this respect, the framework’s most important contribution is not only dataset expansion, but the attempt to preserve **cultural relevance**, **safety labels**, **translation quality**, and **adversarial coverage** simultaneously [2508.01710].

The **cross-lingual safety consistency filter** is particularly significant because it provides a way to filter multilingual data **without needing a non-English safety model**. At the same time, the paper notes a limitation: if back-translation introduces errors, valid samples may be discarded. A similar design trade-off appears in the adaptation stage. Adaptation is performed in English because open LLMs do not perform well on non-English text, but this also means that cultural transfer is mediated through English before being rendered into the target language [2508.01710].

The paper’s benchmarking of public open models further motivates the framework. It reports that **open LLMs are generally less safe in non-English prompts than in English** when used as guard models on the CultureGuard harmful prompt set. This degradation is stated to be especially severe for **Hindi** and **Japanese**. Although safety generally improves with increasing model size, the paper argues that **multilingual safety is not solved by scaling alone**. The notable exception identified is **Gemma-2 (9B/27B)**, described as the safest among the evaluated public models, with multilingual safety roughly on par with English performance [2508.01710].

This empirical pattern supports a common misconception correction: CultureGuard is not simply a translated safety benchmark plus LoRA fine-tuning. Rather, it is a dataset-and-model pipeline built around the proposition that safety labels can drift under cultural adaptation and machine translation, and that these shifts must be explicitly modeled and filtered if multilingual guard models are to remain reliable [2508.01710].

## 6. Position in later literature and regional specialization

Subsequent work treats CultureGuard as a major prior baseline in culturally aware multilingual safety. **IndicGuard** describes CultureGuard as “a culturally-aware multilingual safety dataset and guard model tailored for multilingual moderation settings” and uses it as the main comparison point across **English, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Punjabi, Tamil, Telugu, and Urdu**. In the aggregate **Combined** setting, IndicGuard reports **Macro F1** values of **0.8190** for **User Safety** and **0.8261** for **Response Safety** for CultureGuard, versus **0.8808** and **0.8847** for IndicGuard, corresponding to improvements of **0.0618** and **0.0586** respectively [2606.22841].

The same later work also characterizes CultureGuard as a system developed to move “beyond conventional translation-based safety pipelines” by introducing **culturally grounded safety annotations across multiple languages** and a guard model that distinguishes **culturally contextualized benign content** from **genuinely harmful content**. In that framing, CultureGuard functions both as a conceptual predecessor and as an empirical baseline for region-specific successors [2606.22841].

A closely related regional extension is **SEA-Guard**, which explicitly presents itself as a **CultureGuard-style safeguard system** grounded in Southeast Asian cultural contexts. SEA-Guard is described as a regional specialization of the broader CultureGuard idea, replacing translation-centric multilingual safety with a **large synthetic, culturally annotated, multilingual safety corpus** and SEA-specific guard models [2602.01618]. This suggests that CultureGuard’s most durable legacy is methodological: it established culturally aware multilingual safeguarding as a data-centric design problem rather than merely a transfer-learning problem.

A distinct but related trajectory appears in **Guard Vector**, which approaches multilingual guardrails through **task-vector composition** rather than dataset construction. Guard Vector transfers a safety direction in parameter space into target-language models and reports extension to **Chinese, Japanese, and Korean** without additional training or target-language labels [2509.23381]. This suggests a bifurcation in later research: one line extends CultureGuard’s culturally grounded dataset-and-model pipeline, while another explores weight-composition methods for multilingual guardrail portability.

Taken together, these later developments position CultureGuard as an early, explicit attempt to close the gap between English-centric safety systems and multilingual, culturally robust moderation. Its enduring significance lies in the combination of **cultural adaptation**, **translation-aware safety preservation**, **multilingual jailbreak coverage**, and **guard-model training** within a single reproducible pipeline [2508.01710].

Source: https://www.emergentmind.com/topics/cultureguard