Saudi Dialect Instruction Dataset

• Saudi Dialect Instruction Dataset is a supervised corpus that maps instruction prompts to tailored outputs in Hijazi and Najdi Arabic varieties.
• It employs synthetic data generation via API-assisted prompting with a strict 50/50 split and stratified train/dev/test sampling over dialect, topic, and length.
• Evaluations reveal that incorporating explicit dialect tags significantly improves language model controllability and reduces Modern Standard Arabic leakage.

A Saudi Dialect Instruction Dataset is a supervised corpus designed to map prompts or instructions to outputs in Saudi Arabic varieties. In the current arXiv literature, the clearest direct instantiation is the private synthetic corpus introduced for “Saudi-Dialect-ALLaM,” which targets Hijazi and Najdi generation through single-turn instruction–response pairs (Barmandah, 19 Aug 2025). Surrounding this core example is a broader ecosystem of adjacent resources: pan-Arab instruction datasets with Saudi subsets, cross-dialect phrase collections that include Saudi renderings, Saudi-dialect evaluation benchmarks, multimodal dialect-adaptation sets, and Gulf-oriented translation corpora (Alwajih et al., 28 Feb 2025). Taken together, these works define the topic less as a single canonical dataset than as a family of resources with sharply different assumptions about dialect granularity, authenticity, supervision format, and release policy.

1. Corpus identity and definitional boundaries

The most explicit “Saudi Dialect Instruction dataset” in the literature is the privately curated corpus used to LoRA-tune ALLaM-7B-Instruct-preview for Saudi dialect generation (Barmandah, 19 Aug 2025). That paper describes the resource as a “synthetic instruction–response corpus” or “synthetic instruction–response dataset,” created specifically to address the underrepresentation of Saudi dialects and the tendency of Arabic LLMs to drift toward Modern Standard Arabic (MSA). Its scope is narrow but unambiguous: two dialect labels, Hijazi and Najdi, with a total of 5,466 instruction–response pairs and a strict 50/50 split (Barmandah, 19 Aug 2025). This suggests, though the paper does not state the counts explicitly, 2,733 examples per dialect.

A recurrent boundary problem in the literature is that “Saudi dialect data” and “Saudi dialect instruction data” are not synonymous. Several resources contain Saudi material without being instruction-tuning corpora. “Absher” is a Saudi-dialect benchmark composed of multiple-choice questions rather than open-ended instruction–response supervision (Al-Monef et al., 14 Jul 2025). “ArabicDialectHub” includes Saudi translations for a shared cross-dialect phrase inventory, but it is presented as a pedagogical parallel phrase resource rather than an instruction dataset (Lahlou, 30 Jan 2026). “Dallah” contains a Saudi Arabia multimodal dialect subset used in instruction tuning, but it is image-grounded and country-level rather than a standalone Saudi text-only corpus (Alwajih et al., 2024). Conversely, “CIDAR” is an instruction dataset, but the paper explicitly states that its localization remained “mostly written in MSA, without incorporating multiple dialects,” so it is not a Saudi dialect dataset in the strict sense (Alyafeai et al., 2024).

A second boundary issue concerns dialect granularity. The dedicated Saudi dataset in “Saudi-Dialect-ALLaM” is explicitly Hijazi/Najdi (Barmandah, 19 Aug 2025). By contrast, several adjacent resources use only the country label “Saudi Arabia” or “Saudi,” without finer distinctions such as Hijazi, Najdi, Eastern, or Southern Saudi (Alwajih et al., 28 Feb 2025). This difference matters because country-level Saudi labeling and subdialectal Saudi control are methodologically distinct objectives.

2. Documented design of the dedicated Saudi instruction corpus

The record-level schema in the Saudi-Dialect-ALLaM corpus contains at minimum three fields: an instruction, a “dialect-pure response,” and a categorical dialect label (Barmandah, 19 Aug 2025). For supervised fine-tuning, each example is formatted as a simple instruction-tuning text pair:

Instruction: [user prompt] Response: [dialectal output]

The corpus is strictly balanced across Hijazi and Najdi to “prevent dialect–topic confounds” (Barmandah, 19 Aug 2025). The paper also reports that train/dev/test splits were constructed by 80/10/10 stratified sampling over three metadata dimensions: dialect, topic, and length. Topic coverage is organized into 18 categories, although the full taxonomy is not printed in the paper. The top five most common topics are shopping_markets, work_careers, education_learning, health_fitness, and technology_gadgets, which together account for “roughly 55% of the dataset” (Barmandah, 19 Aug 2025). Length is binned into short, medium, and long, but no per-bin counts are provided.

A notable feature is the comparison between two formatting variants built from the same underlying examples. In the Dialect-Token variant, an explicit tag is prepended to the instruction. The paper gives two inconsistent notations for these tags: <DIALECT=HIJAZI> / <DIALECT=NAJDI> in one section, and <HIJAZI> / <NAJDI> in another (Barmandah, 19 Aug 2025). In the No-Token variant, the same examples are used but the explicit tag is removed at formatting time. Conceptually, the difference is not about corpus content but about whether dialect identity is surfaced as an explicit conditioning signal.

The paper characterizes the responses as “dialect-pure,” but it does not define a formal annotation rubric for purity (Barmandah, 19 Aug 2025). It likewise does not specify whether the dialect label exists as a structured field in the released training files, because the dataset itself is not released. The design is therefore well specified at a high level and underspecified at the serialization level.

3. Construction, curation, and annotation regime

The dedicated Saudi corpus is synthetic rather than naturally occurring or fully human-authored. The authors state that the instruction–response pairs were “synthesized via API-assisted prompting and then curated,” followed by cleaning, balancing, and training-time formatting (Barmandah, 19 Aug 2025). Concrete preprocessing steps include dropping “unknown or ambiguous dialect entries,” enforcing exact 50/50 balance, tokenizing with the ALLaM-7B-Instruct-preview tokenizer, and truncating or packing sequences to a maximum of 2048 tokens (Barmandah, 19 Aug 2025).

What the paper does not provide is equally important. It does not disclose the identity of the API provider, the exact prompts used to synthesize the data, the exact source model or models, a detailed annotation workflow, inter-annotator agreement, or a formal validation study of the dataset labels (Barmandah, 19 Aug 2025). The only human annotation protocol described in detail pertains to model-output evaluation, not dataset creation. Quality control is therefore documented as curation rather than as a full annotation audit.

This pattern contrasts with adjacent resources. “Palm” is a year-long, community-driven dataset of 17,411 human-created instruction pairs covering all 22 Arab countries, with Saudi Arabia represented by 1,299 total examples and 296 dialectal examples (Alwajih et al., 28 Feb 2025). “ArabicDialectHub” combines LLM generation with review by five native Arabic speakers, but its limitation section explicitly states that Syrian, Emirati, and Saudi translations lack native-speaker verification (Lahlou, 30 Jan 2026). “Dallah” created a Saudi Arabia multimodal dialect subset by assigning translated MSA items to native professional translators from Saudi Arabia, but it does not give a standalone Saudi annotation protocol beyond that description (Alwajih et al., 2024). “CIDAR” used around 12 contributors for cultural localization, yet explicitly acknowledges that its data remained mostly in MSA rather than multiple dialects (Alyafeai et al., 2024).

A plausible implication is that current Saudi dialect instruction resources occupy three distinct construction regimes: synthetic-and-curated dedicated corpora, human-authored pan-Arab corpora with Saudi subsets, and repurposable Saudi-containing resources whose original objective was not instruction tuning.

4. Related datasets and non-equivalent resources

Several datasets are often adjacent to the Saudi Dialect Instruction Dataset problem without being direct substitutes.

Resource	Saudi relevance	Data form
“Saudi-Dialect-ALLaM” (Barmandah, 19 Aug 2025)	Dedicated Hijazi/Najdi corpus	5,466 synthetic instruction–response pairs
“Palm” (Alwajih et al., 28 Feb 2025)	Saudi subset: 1,299 total, 296 dialect	Human-created instruction–response pairs
“ArabicDialectHub” (Lahlou, 30 Jan 2026)	Saudi renderings for 552 shared phrase concepts	Cross-dialect phrase resource
“Absher” (Al-Monef et al., 14 Jul 2025)	Regional Saudi benchmark	18,564 multiple-choice questions
“Dallah” (Alwajih et al., 2024)	Saudi Arabia subset: 784	Multimodal dialect instruction tuning
“CIDAR” (Alyafeai et al., 2024)	Pan-Arab cultural alignment, mostly MSA	10,000 instruction–output pairs

“Palm” is the strongest public Saudi-adjacent instruction dataset in the sense that it explicitly contains Saudi country-level instruction–response pairs and a dialectal Saudi subset, while remaining fully pan-Arab rather than Saudi-only (Alwajih et al., 28 Feb 2025). The paper does not, however, name narrower Saudi subvarieties. “ArabicDialectHub” is structurally closer to a multilingual phrase table than to an instruction corpus: Saudi appears once for each of the 552 aligned phrase entries, along with metadata such as difficulty, category, romanization, and usage notes (Lahlou, 30 Jan 2026). “Absher” is not training data at all, but an evaluation benchmark spanning Meaning, True/False, Fill-in-the-Blank, Contextual Usage, Cultural Interpretation, and Location Recognition across Central, Western, Southern, Northern, Eastern, and General Saudi Terms (Al-Monef et al., 14 Jul 2025).

Two other resources are relevant mainly by repurposing. “OSN-MDAD” provides 15,000 English–Gulf parallel tweet translations within a four-dialect social-media MT dataset, but it does not identify Saudi as a separate dialect and is not an instruction dataset (Alzamzami et al., 2023). “SD-QA” includes Saudi Arabia (SAU) as one of seven Arabic spoken varieties, with 3 Saudi dev speakers and 7 Saudi test speakers, but it is a spoken extractive QA benchmark built from TyDi-QA rather than an instruction-following corpus (Faisal et al., 2021). These datasets are useful for Saudi-adjacent supervision, evaluation, or augmentation, but they do not solve the text instruction-tuning problem directly.

5. Training use, conditioning strategies, and evaluation

The dedicated Saudi corpus is used for supervised fine-tuning under a causal language modeling objective. The paper gives the loss as

$$\mathcal{L}_{CE} = - \frac{1}{N}\sum_{i=1}^N \log p_\theta(y_i \mid y_{<i}, x)$$

where the response tokens are predicted autoregressively conditioned on the instruction context $x$ (Barmandah, 19 Aug 2025). Fine-tuning uses LoRA adapters with rank $r = 32$, scaling factor $\alpha = 64$, dropout $p = 0.1$, 15 epochs, batch size 2, gradient accumulation 8, BF16, and maximum sequence length 2048 (Barmandah, 19 Aug 2025).

Evaluation is tightly coupled to the held-out Saudi test set derived from the same corpus. Automatic assessment combines an external dialect classifier, reference-based fidelity metrics, and diversity measures. The dialect classifier is the MARBERTv2 Arabic Written Dialect Classifier over MAGHREB / LEV / MSA / GLF / EGY, with GLF used as “a practical proxy for Saudi usage” (Barmandah, 19 Aug 2025). The paper is explicit that this is only an approximation and that the classifier “does not perfectly separate Hijazi from Najdi.” Dialect control is operationalized as

$$\text{Saudi\%} = \frac{1}{N} \sum_{i=1}^{N} \mathbf{1}\{\hat{y}_i = \text{GLF}\} \times 100$$

with separate measures for MSA leakage and low-confidence outputs (Barmandah, 19 Aug 2025).

Variant	Dialect control	Fidelity
ALLaM-7B base	47.97% Saudi; 32.63% MSA leak	21.27 chrF++; 0.6796 BERTScore F1
No-Token LoRA	80.50% Saudi; 9.26% MSA leak	23.70 chrF++; 0.7377 BERTScore F1
Dialect-Token LoRA	84.21% Saudi; 6.21% MSA leak	24.80 chrF++; 0.7386 BERTScore F1

These results support the paper’s claim that explicit dialect tags improve controllability and reduce MSA leakage (Barmandah, 19 Aug 2025). The gains are strongest in dialect control and text-reference alignment rather than uniformly across all diversity metrics. Human evaluation further reports that ALLaM-LoRA-Token achieved 68.83% on dialect correctness, 74.83% on fluency/naturalness, and 91.50% on task adherence, although the paper contains an internal inconsistency about the sample size, stating both “40 prompts per model” and “only three annotators and 100 prompts per model” in different places (Barmandah, 19 Aug 2025).

Findings from adjacent papers reinforce the same general problem. “Palm” reports that dialectal consistency remains below roughly 10% across all tested LLMs, with models tending to answer in MSA about 90% of the time even when prompted in dialectal Arabic (Alwajih et al., 28 Feb 2025). “Absher” shows notable performance gaps on Saudi-dialect understanding, especially for cultural inference and contextual understanding (Al-Monef et al., 14 Jul 2025). “Dallah” evaluates dialectal multimodal responses using Dialect Authenticity and Content Accuracy, including a Saudi row, which suggests that explicit dialect evaluation has become a recurring requirement even outside text-only instruction tuning (Alwajih et al., 2024).

6. Limitations, controversies, and requirements for future corpora

The central limitation of the dedicated Saudi instruction dataset is non-release. The authors state that they do not release the raw dataset, model weights, or LoRA adapters, and instead release training, evaluation, and inference code together with a datasheet containing schema, topic taxonomy, cleaning rules, and aggregate statistics (Barmandah, 19 Aug 2025). The reason given is “provider terms and redistribution risk.” This makes the corpus methodologically visible but empirically non-replicable at the example level.

Its second limitation is representational scope. The corpus is single-turn, synthetic, and “dialect-pure,” and it “does not include multi-turn dialogue, natural conversations, or spontaneous interaction patterns” (Barmandah, 19 Aug 2025). The paper also acknowledges modest size, restricted topical coverage, and reduced coverage of less common dialectal phenomena, idiomatic expressions, and low-frequency lexical items. These constraints directly limit general-purpose instruction-following behavior.

A third limitation is dialect ontology. Although the dedicated corpus distinguishes Hijazi and Najdi, many related Saudi-containing datasets do not move beyond the country label “Saudi Arabia” or “Saudi” (Alwajih et al., 28 Feb 2025). “ArabicDialectHub” does not discuss internal Saudi regional variation at all; its dialect label is simply “Saudi,” and the Saudi translations lack native-speaker verification (Lahlou, 30 Jan 2026). “Absher” is regionally richer, but it is an evaluation benchmark rather than an instruction corpus (Al-Monef et al., 14 Jul 2025). “CIDAR” provides culturally aligned Arabic instruction data, yet the paper explicitly states that dialects were not incorporated (Alyafeai et al., 2024). The result is a fragmented landscape in which Saudi specificity, dialectal depth, and instruction format seldom co-occur.

A persistent misconception is that any dataset containing Saudi Arabic automatically constitutes a Saudi Dialect Instruction Dataset. The literature shows otherwise. Phrase tables, MCQ benchmarks, multimodal adaptation sets, Gulf translation corpora, and spoken QA resources are all reusable, but they encode different supervision signals and different notions of dialect control (Lahlou, 30 Jan 2026). A production-grade Saudi instruction dataset would therefore require, at minimum, the properties that current resources only partially satisfy: explicit Saudi-native validation, transparent schema and prompting records, broader topical and conversational coverage, release conditions that permit independent verification, and dialect labeling that resolves country-level Saudi from subvarieties such as Hijazi and Najdi. This suggests that the field is still in a formative stage, with the dedicated corpus in (Barmandah, 19 Aug 2025) functioning more as a proof of feasibility than as a public canonical benchmark.