SEALGuard: Multilingual Safety Guardrail System

• SEALGuard is a multilingual guardrail system designed to filter unsafe prompts in LLMs across low-resource Southeast Asian languages.
• It employs Low-Rank Adaptation (LoRA) on the SeaLLM backbone to achieve high defense success rates, precision, and parameter efficiency.
• Benchmarking with SEALSBench shows SEALGuard's superior performance over English-centric filters, ensuring robust and scalable safety alignment.

SEALGuard is a multilingual guardrail system engineered to address safety alignment challenges in LLM-powered conversational agents, particularly for low-resource Southeast Asian languages. It integrates a dedicated adaptation layer atop the SeaLLM multilingual foundation model to classify and filter unsafe and jailbreak prompts, thereby mitigating vulnerabilities present in existing English-centric guardrails. SEALGuard achieves this using Low-Rank Adaptation (LoRA) and leverages a large-scale benchmark, SEALSBench, to empirically validate its effectiveness on ten languages. The architecture, evaluation metrics, experimental results, and ablation studies consistently demonstrate marked improvements in defense success rates, precision, and robustness across diverse language settings (Shan et al., 11 Jul 2025).

1. Motivations and System Overview

Existing guardrail approaches, such as LlamaGuard, exhibit high detection accuracy for English unsafe prompts (≈95%) but experience substantial performance degradation (9–18% Defense Success Rate (DSR) loss) when presented with prompts in Southeast Asian languages. Empirical analyses reveal straightforward adversarial strategies—such as translating unsafe content or jailbreak attempts—that can bypass English-oriented filters, posing real-world vulnerabilities as Southeast Asia comprises numerous under-resourced languages with limited moderation technologies. SEALGuard directly targets this multilingual safety gap to support inclusivity and robust moderation.

The SEALGuard inference pipeline consists of five steps:

1. Guardrail Chat Template: Prompts are wrapped with instructions to output “safe” or “unsafe,” explicitly listing unsafe content categories using a dedicated token boundary.
2. Tokenization and Encoding: The prompt is tokenized with SentencePiece BPE, then fed through frozen SeaLLM embeddings and 28 transformer decoder layers.
3. LoRA Module Injection: Trainable LoRA modules are introduced into selected SeaLLM weight matrices (word embeddings, self-attention projections, feed-forward layers).
4. Greedy Decoding: Only the first token after the template is decoded; this single token (“safe” or “unsafe”) determines input fate.
5. Filtering Enforcement: “Unsafe” prompts are blocked from downstream processing; “safe” prompts are allowed as input to the LLM.

This pipeline is architected to minimize system latency and avoid spurious content exposure.

2. LoRA Adaptation Methodology

SEALGuard utilizes LoRA modules to confer safety classification capability atop the pretrained multilingual representations of SeaLLM while retaining its full language generalization. For a frozen pretrained weight matrix $W_0 \in \mathbb{R}^{h\times h}$, LoRA replaces its contribution with:

$W' = W_0 + B A$

where $A \in \mathbb{R}^{h\times r}$ and $B \in \mathbb{R}^{r\times h}$ are the trainable low-rank factors and $r\ll h$ (empirically $r=8$).

• Embedding Layer Adaptation: The embedding matrix $W_E \in \mathbb{R}^{v\times h}$ (vocab size $v=152\,064$, embedding dim $h=3\,584$) receives LoRA augmentation. All SeaLLM parameters remain frozen except for LoRA modules.
• Transformer Layer Adaptation: Each self-attention projection and FFN matrix of dimension $h\times h$ is similarly updated.
• Parameter Efficiency: Only LoRA factors ($W' = W_0 + B A$0, $W' = W_0 + B A$1) are trained, with minimal new parameters added, enabling scalability and rapid adaptation to safety-classification objectives.

This adaptation strategy enables SEALGuard to specialize for safety alignment without high resource requirements or catastrophic forgetting of pre-existing multilingual competence.

3. SEALSBench Dataset and Benchmark Protocol

SEALSBench is constructed as a multilingual safety alignment dataset, purpose-designed for Southeast Asian LLM moderation research:

• Composition: 266,444 prompts comprised of ten languages (English plus Zho, Ind, Vie, Tha, Khm, Lao, Msa, Mya, Tgl) with three labels: safe (169,433), unsafe (80,601), and jailbreak (16,410).
• Curation and Annotation:
  • Unsafe prompts were selected from BeaverTails’ ten categories and jailbreak-oriented datasets (Do-Not-Answer, CatQA, AdvBench, Forbidden Questions).
  • Safe prompts derived from Alpaca (standard innocuous inquiries).
  • All prompts originated in English and were automatically translated via Google Translate API; translation IDs ensure one-to-one correspondence across languages.
  • Labels were preserved across translations without manual relabeling, as unsafe/jailbreak/safe status reliably transfers.

This benchmark enables rigorous, consistent evaluation across diverse language boundaries and safety contexts.

4. Evaluation Metrics and Definitions

SEALGuard is evaluated with the following metrics:

• Defense Success Rate (DSR):

$W' = W_0 + B A$2

$W' = W_0 + B A$3 denotes correctly blocked unsafe or jailbreak prompts; $W' = W_0 + B A$4 represents unsafe prompts missed.

• Precision:

$W' = W_0 + B A$5

$W' = W_0 + B A$6 indicates safe prompts incorrectly blocked.

• F1-Score:

$W' = W_0 + B A$7

These metrics emphasize both safety efficacy and false positive minimization, critical for practical deployment.

5. Comparative Experiments with Existing Guardrails

On the full SEALSBench multilingual test set:

• SEALGuard (SeaLLM-7B + LoRA) achieved DSR 97.2%, Precision 98.9%, F1 98.1%.
• LlamaGuard-3-8B obtained DSR 49.6%, Precision 95.8%, F1 63.6%.
• OpenAI Moderation and LlamaGuard-3-1B performed similarly or worse (DSR ≈44–45%, F1 < 50%).

SEALGuard outperformed LlamaGuard-8B by +48 percentage points DSR, +3 points precision, and +34 points F1.

Breakdown by category:

• Unsafe-only: SEALGuard DSR 97% vs. LlamaGuard-8B 49.6%.
• Jailbreak: SEALGuard DSR 97% vs. LlamaGuard-8B 41.0%.

Breakdown by language reveals that SEALGuard exceeds 97% F1 on all ten languages, while LlamaGuard-8B ranges from 40–60%.

System	DSR	Precision	F1
SEALGuard (SeaLLM-7B + LoRA)	97.2%	98.9%	98.1%
LlamaGuard-3-8B	49.6%	95.8%	63.6%

The marked improvement indicates SEALGuard’s ability to close the safety gap left by English-focused moderation, especially under adversarial translation scenarios.

6. Ablation Studies on Adaptation and Model Size

Ablative experiments clarify contributing factors to performance:

Variant	DSR	Precision	F1
SeaLLM-7B + LoRA	97.2%	98.9%	98.1%
SeaLLM-1.5B + LoRA	96.5%	98.2%	97.3%
SeaLLM-7B + NeMo Guardrails	49.9%	77.2%	60.6%
SeaLLM-1.5B + NeMo Guardrails	89.5%	34.9%	50.2%
SeaLLM-7B (no adaptation)	15.1%	89.5%	25.8%
SeaLLM-1.5B (no adaptation)	18.5%	60.1%	28.3%

• LoRA adaptation is critical: removal results in F1 below 30%, whereas LoRA maintains F1 above 97%, regardless of model scale.
• Model size is secondary: 1.5B parameters yields near-equivalent performance to 7B when LoRA is present.
• Off-the-shelf NeMo Guardrails underperform compared to LoRA, indicating the adaptation’s importance for task specialization.

A plausible implication is that the core multilingual knowledge encoded in the frozen SeaLLM backbone, paired with LoRA adaptability, suffices for high-accuracy safety alignment across resource-constrained languages.

7. Limitations and Prospective Advancements

Limitations

• Translation Artifacts: Automated prompt translations may introduce artifacts or distributional drift, impacting less-resourced languages.
• Coverage: SEALSBench addresses only ten languages; many Southeast Asian and global languages remain untested.
• Dynamic Threats: Static input-side guardrails cannot anticipate or block every emergent jailbreak pattern (e.g., novel obfuscation, role-play attacks).

Future Research Directions

• Expansion to Additional Languages: Extend coverage to further low-resource languages (e.g., Rohingya, Sinhalese) and dialects.
• Continual Learning: Integrate online or active-learning paradigms so adaptation can respond to new attack patterns dynamically.
• Hybrid Defense Mechanisms: Combine input filtering with output or post-hoc safety checks for layered protection.
• Minimized Labeled Data Dependence: Investigate unsupervised/self-supervised safety alignment to lower annotation costs.
• Human-in-the-loop Updates: Use adversarial data and expert annotations to continually refine LoRA modules and safety policies.

SEALGuard demonstrates that lightweight, parameter-efficient LoRA adaptation of a multilingual foundation model is an effective approach to multilingual, safe prompt filtering, closing existing safety gaps with near-perfect defense rates in a resource-efficient, scalable manner (Shan et al., 11 Jul 2025).