HausaSafety Framework

• HausaSafety is a framework that defines curated datasets, methodologies, and auditing paradigms for detecting and mitigating offensive Hausa content.
• It employs traditional ML pipelines, neural architectures, and transformer fine-tuning to benchmark and improve safety alignment in digital ecosystems.
• Robust evaluations using precision, recall, F1 scores, and adversarial testing reveal linguistic, temporal, and cultural challenges in Hausa content moderation.

HausaSafety encompasses the theoretical frameworks, datasets, methodologies, and auditing paradigms required for robust detection and mitigation of offensive, hateful, and harmful content in Hausa language digital ecosystems, as well as the measurement and improvement of safety alignment for Hausa in LLMs. Its remit covers both content-moderation for user-generated Hausa social media and system-level safety evaluation in cross-lingual AI, reflecting recent large-scale corpus creation, adversarial benchmarking, and alignment research.

1. Dataset Resources and Corpus Engineering

The foundation of HausaSafety is the curation of large, diverse, and manually annotated datasets capturing the complex landscape of Hausa digital text. Pioneering resources include:

• Social Stream and News/Blog Corpora: Comprehensive datasets from Twitter (12,038 tweets, ~95.5k tokens), Facebook, BBC Hausa, VOA Hausa, and indigenous blogs, capturing both informal (slang, memes, emoji, code-switching) and formal (news, essays) registers (Inuwa-Dutse, 2021).
• Parallel Hausa–English Alignments: 310 aligned tweet pairs, supporting machine translation and cross-lingual consistency checks (Inuwa-Dutse, 2021).
• Domain-Specific Offensive/Hate Datasets: Offensive and hate-speech subsets (e.g., 4,467 Hausa tweets labeled on a three-way taxonomy: hate, offensive, neither) and large-scale collections (~15,000 posts labeled for offense, ~8,000 for threat), annotated with inter-annotator agreement (Cohen’s κ = 0.78) (Adam et al., 2023, Aliyu et al., 2024).
• Threat/Incitement and Misinformation Annotation: Expanded label sets for incitement and misinformation allow differentiation between genuine threats, slurs, and false claims.

Key preprocessing includes language/dialect filtering, aggressive de-duplication, normalization (handling tokenization, spelling and diacritics), stopword stripping, and code-mixing flags (Adam et al., 2023, Inuwa-Dutse, 2021, Aliyu et al., 2024).

2. Detection Architectures and Hausa-Specific Modeling

Detection systems for HausaSafety leverage both traditional and deep learning models:

• Supervised ML Pipelines: Logistic Regression, SVM, Naive Bayes, Random Forest, and particularly XGBoost (best F₁ = 0.86 on offensive detection) (Adam et al., 2023), trained with TF–IDF representations (notably trigrams for high-precision idiom capture), character n-grams, and code-mixing features.
• Neural Models: Multilayer Perceptron and 1D CNN architectures using pre-trained embeddings (Word2Vec on 500M-token corpus) and convolutional filters over trigrams (Adam et al., 2023).
• Transformer Fine-tuning: Benchmarking on XLM-RoBERTa, mBERT, morit/XLM-T, and Davlan/Afriberta models, the last yielding highest Hausa accuracy (0.85) without explicit Hausa pretraining—underscoring the value of domain adaptation from Nigerian Twitter data (Aliyu et al., 2024).
• Tokenization and Data Augmentation: Special attention to Hausa orthography, synonym-based EDA, and back-translation (Hausa→English→Hausa) is recommended (Inuwa-Dutse, 2021).

Train/test splits adhere to 80/20 stratified protocols, with robust validation including 5-fold cross-validation for hyperparameter selection and unseen test splits for final evaluation (Inuwa-Dutse, 2021, Adam et al., 2023, Aliyu et al., 2024).

3. Evaluation Metrics, Benchmarking, and Error Analysis

Performance tracking in HausaSafety emphasizes:

• Precision, Recall, and F1-score: Defined as

$$\mathrm{Precision} = \frac{TP}{TP+FP}\,,\quad \mathrm{Recall} = \frac{TP}{TP+FN}\,,\quad \mathrm{F}_1 = 2\frac{\mathrm{Precision}\times\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

with balanced classes for production-grade reliability (Adam et al., 2023, Inuwa-Dutse, 2021, Aliyu et al., 2024).

• Class Imbalance: Hateful content is rare (~7%), offensive non-hate ~23%, and non-offensive ~70% in Twitter datasets (Aliyu et al., 2024).
• Model Comparison: XGBoost (trigram) and Afriberta-Large set the state-of-the-art in automated content moderation, but even top models (F₁ ≈ 0.86) exhibit persistent error modes (Adam et al., 2023, Aliyu et al., 2024).
• Cross-lingual Drift: Google Translate + English classifiers underperform drastically (F₁ ≈ 0.55), failing on idioms and Engausa code-mix (Adam et al., 2023).
• Error Analysis: Major pitfalls include classificatory confusion from proverbs, code-switching, dialectal variations, mislabeled banter, and the reporting of real events without malicious intent (Adam et al., 2023, Aliyu et al., 2024).

4. Safety Alignment in LLMs: Adversarial Evaluation and Robustness

Recent adversarial benchmarks expose severe vulnerabilities in Hausa safety alignment for LLMs:

• LSR Benchmark and Refusal Centroid Drift (RCD): Matched English/Hausa prompts probe model refusal rates. The metric

$\mathrm{RCD}(L) = 1 - \frac{R_L}{R_{EN}}$

quantifies loss of English refusal behavior in a target language, with Hausa refusal rates typically falling to 35–55% compared to ~90% in English. Igala shows the highest drift (RCD=0.55) (Faruna, 27 Feb 2026).

• HausaSafety Adversarial Dataset: 60 base prompts × four scenario classes (financial crime, social engineering, cultural violence, information ops), rendered bilingually with temporal transformations (past, present, future, displacement), yielding 480 unique test cases per language (Said et al., 31 Dec 2025).
• Complex Interference and Safety Pockets: Safety is influenced by both linguistic and temporal framing—a “complex interference.” For example, Claude 4.5 Opus is safer in Hausa (45.0%) than English (36.7%), but temporal cues yield 3.7× greater safety in future-tense (57.2%) than past-tense (15.6%) (Said et al., 31 Dec 2025).
• Statistical Rigor: Two-way ANOVA reveals strong main and interaction effects (p<0.001), with up to 9.2× disparity in safety rates between best and worst configurations (Said et al., 31 Dec 2025).

Model	English Safety Rate	Hausa Safety Rate	Gap
Claude 4.5 Opus	36.7%	45.0%	+8.3%

Temporal Frame	Safety Rate (SR)	Attack Success Rate (ASR)
Past	15.6%	84.4%
Present	40.0%	60.0%
Future	57.2%	42.8%
Displacement	28.1%	71.9%

These tables encapsulate nontrivial safety asymmetries by language and temporal frame (Said et al., 31 Dec 2025).

5. Cultural and Linguistic Challenges

HausaSafety research consistently foregrounds the crucial role of cultural context, dialect, and code-switching:

• Idioms and Proverbs: Insults or threats may be veiled in euphemisms (e.g., “idan maye ya manta”) or indirect references that models misclassify (Adam et al., 2023, Aliyu et al., 2024).
• Code-mixing and Dialect: Frequent English insertions (“Engausa”), regional spelling variations, and orthographic diversity complicate feature extraction and wordpiece tokenization (Aliyu et al., 2024, Inuwa-Dutse, 2021).
• Contextual Nuance: Terms like “dan iska” or “shegiya” shift offensiveness by context, challenging universal tagging. Playful banter is frequently mischaracterized, while political and religious topics drive more offensive/threatening language (Adam et al., 2023).

The most problematic false negatives arise from subtle or context-dependent offense, while false positives may reflect failure to discern genuine hostility from repartee among acquaintances (Adam et al., 2023, Aliyu et al., 2024).

6. Safety-Centric Moderation, Alignment Protocols, and Mitigation

The HausaSafety paradigm encompasses both operational moderation and the calibration of AI refusal mechanisms:

• Moderation Workflows: Automated flagging via best-performing ML models, followed by human review (context, topic, sentiment) and graduated response (warning → shadowban → suspension) (Adam et al., 2023).
• Stakeholder Engagement: Regular updates to insult lexicons by community experts, audits by Hausa-speaking moderators, and policy support for safety module integration (Adam et al., 2023).
• Invariant Alignment: To address “safety pockets” and context-dependent failures, the invariant alignment principle mandates that a refusal classifier satisfy

$$\forall x,~\forall T \in \mathcal{T}: S_\theta(x) = S_\theta(T(x))$$

across all translation and temporal reframings, enforced via a penalty in the alignment loss:

$$\mathcal{L}_{\text{inv}} = \sum_{T \in \mathcal{T}} \| S_\theta(x) - S_\theta(T(x)) \|^2$$

Optimizing $$\min_\theta \mathcal{L}_{\text{safe}} + \lambda \mathcal{L}_{\text{inv}}$$, this approach seeks semantic refusal invariance beyond surface-form patching (Said et al., 31 Dec 2025).

Targeted interventions include real-time “nudge” warnings, in-app glossaries for offensive terms, and community-defined lists of reclaimed expressions used in benign contexts (Adam et al., 2023).

7. Open Challenges, Recommendations, and Future Directions

Critical open issues include:

• Persistent Class Imbalance: Hateful speech remains rare, constraining recall; future work should augment data by leveraging lexicon and bootstrapped semi-supervised expansion (Aliyu et al., 2024).
• Domain and Temporal Coverage: Much data is Twitter-centric; broader platform sampling and temporal slice benchmarking are necessary for generalization (Adam et al., 2023, Inuwa-Dutse, 2021).
• Robust Cross-Lingual Evaluation: Systematic benchmarking (leveraging RCD and similar metrics) across Hausa and other local languages establishes cross-language comparability and exposes context-specific gaps (Faruna, 27 Feb 2026, Said et al., 31 Dec 2025).
• Community-Driven Tools and Open Resources: Public corpus and model releases under open governance, with reproducibility protocols, are essential for sustainable advancements (Inuwa-Dutse, 2021, Aliyu et al., 2024).
• Algorithmic Fairness and Contextualization: Including real-world stakeholder feedback in model tuning and review processes enhances cultural fit and reduces unintentional harm (Adam et al., 2023).

HausaSafety thus comprises a composite framework for dataset curation, ML model engineering, cultural-linguistic adaptation, and systemic AI safety alignment, underpinned by quantitative benchmarks and a deep commitment to semantic invariance and representativeness in content moderation and LLM safety (Inuwa-Dutse, 2021, Adam et al., 2023, Aliyu et al., 2024, Faruna, 27 Feb 2026, Said et al., 31 Dec 2025).