---
title: 'ALLaM: Arabic-Centric Large Language Models'
url: https://www.emergentmind.com/topics/allam
type: topic
---

# ALLaM: Arabic-Centric Large Language Models

Searching arXiv for recent papers on ALLaM / Allam to ground the article in current literature.
Allam, in current machine-learning literature typically stylized as **ALLaM**, denotes a family of Arabic-centric large language models designed for **Arabic and English** and positioned as infrastructure for **Arabic Language Technologies (ALT)**. The series is presented as a response to the English-centric bias of many frontier LLMs, with an emphasis on **language alignment**, **knowledge transfer at scale**, bilingual competence, and extensive human-preference alignment. Subsequent literature treats ALLaM both as a foundation model family in its own right and as a strong Arabic-native baseline for evaluation in dialect processing, factuality, hallucination analysis, legal reasoning, and retrieval-augmented understanding of historical and religious Arabic [2407.15390].

## 1. Definition, naming, and model family

The name **ALLaM** is explicitly introduced as shorthand for **Arabic Large Language Model**. In relation to the query “Allam,” the foundational paper states that the reference is to **ALLaM**—with capital L’s—and not to the general Arabic word “Allam” [2407.15390].

ALLaM is described as a family of **autoregressive, decoder-only transformer models**. The foundational release reports **7B, 13B, and 70B models initialized from Llama-2 weights**, together with **a 7B model trained from scratch**. A later deployment-oriented evaluation describes the publicly discussed family as including **7B, 13B, 34B, and 70B** parameter scales, with **ALLaM-34B** treated as the most capable public variant at that time and used as the backbone of **HUMAIN Chat**, a closed conversational web service [2407.15390] [2508.17378].

The project is framed as an Arabic-centric alternative to English-first systems. One later paper describes **ALLaM-7B-Instruct-preview** as **the first foundation model developed in Saudi Arabia**, while the UI-level study attributes the family’s introduction to the **Saudi Data and AI Authority (SDAIA)** [2508.13525] [2508.17378]. Across these accounts, the consistent characterization is that ALLaM was built to improve Arabic-language understanding and generation without sacrificing strong English capability.

## 2. Pretraining, tokenization, and alignment strategy

The core technical narrative of ALLaM is bilingual adaptation through **tokenizer expansion**, **staged pretraining**, and **alignment with human preferences**. The foundational paper argues that Llama-2 tokenizes Arabic poorly, often fragmenting Arabic words into characters or byte-fallback units; to address this, the authors train an **Arabic-specific tokenizer** and merge it with the original Llama-2 tokenizer by adding Arabic entries not already present in the original vocabulary. They then compare two initialization schemes for the new token embeddings: random initialization versus initialization from the average of the embeddings of the token’s decomposition under the original tokenizer. The paper reports that this embedding-initialization strategy **dramatically speeds up learning of Arabic** [2407.15390].

Pretraining is staged around a large bilingual mixture. The paper reports about **4 trillion** English tokens from sources such as Dolma-v1 and the Pile, and a curated Arabic corpus of **540B Arabic tokens total**, split into **270B natural Arabic tokens** and **270B translated Arabic tokens**. The translated Arabic portion is important because it is used not merely as data augmentation but to improve **cross-lingual knowledge alignment**. The authors explicitly state that translated Arabic **mitigates catastrophic forgetting in English**, and they report that a roughly **45/55 Arabic/English** ratio is preferred in their ablations [2407.15390].

For the from-scratch 7B model, the reported schedule is **4T English tokens** followed by **1.2T mixed Arabic/English tokens**. The training setup is kept close to Llama-2’s recipe, using **4M tokens per batch**, a maximum learning rate of **\(3 \times 10^{-4}\)** decayed to **\(3 \times 10^{-5}\)** with a cosine schedule, **bf16 mixed precision**, Megatron-LM parallelism, FlashAttention, and an overall compute footprint estimated at **5 million GPU hours** on **128–1024 A100 GPUs** [2407.15390].

Alignment proceeds in two stages. **Supervised fine-tuning (SFT)** uses the **Ultra-Instinct** dataset, reported at **12M samples** in v1 and evenly split between English and Arabic; v2 is smaller but cleaner. During SFT, the model is trained for **3 epochs** with learning rate **\(5 \times 10^{-6}\)** and batch size **1024**, with prompt tokens masked in the loss. Preference alignment uses **Direct Preference Optimization (DPO)** rather than PPO, with batch size **512**, KL penalty **0.1**, learning rate from **\(9 \times 10^{-7}\)** to **\(5 \times 10^{-7}\)** under cosine annealing, and **one epoch** over preference data. The manually verified preference set contains **25,854** triplets and is expanded to **245K samples** by generating multiple rejected candidates per prompt [2407.15390].

## 3. Benchmark performance and deployed conversational behavior

The foundational evaluation presents ALLaM as a high-performing bilingual model family. **ALLaM-70B** is reported to achieve **state-of-the-art performance** on **MMLU Arabic**, **ACVA**, and **Arabic Exams**, and to be the best model on **five of eight** Arabic benchmark sets, including both Arabic MMLU variants, Exams, ETEC, and AraTruthfulQA. On MT-Bench, **ALLaM-70B Instruct** scores **8.19 average in Arabic** and **7.44 average in English**, which the paper uses to support its claims about dialogue quality and bilingual instruction following [2407.15390].

A later deployment-oriented paper evaluates **ALLaM-34B** indirectly through **HUMAIN Chat** rather than through open weights or an API. The study constructs a prompt pack of **23 distinct prompts** across **seven thematic categories**—**MSA**, **Dialect**, **Code-switching**, **Knowledge**, **Reasoning**, **Generation**, and **Safety / security**—and submits each prompt **five times**, yielding **115 outputs**. These outputs are scored by **GPT-5**, **Gemini 2.5 Pro**, and **Claude Sonnet-4** on a **1–5 Likert scale** across **Accuracy**, **Fluency**, **Instruction following**, **Safety**, and **Dialect fidelity** [2508.17378].

The reported category means are high: **Code-switching 4.92**, **Generation 4.92**, **Knowledge 4.77**, **MSA 4.74**, **Reasoning 4.64**, **Safety 4.54**, and **Dialect 4.21**. Safety subcategories are reported at **4.20** for **Prompt Injection**, **Jailbreak**, and **Data Exfiltration**, with effectively zero variance across runs. The same study reports response latency of roughly **1–3 seconds** and interprets the adversarial safety results as stable refusal behavior. Dialect performance is more uneven: the paper reports approximate overall dialect scores of about **3.8** for **Najdi**, **Hijazi**, and **Egyptian**, **3.3** for **Moroccan**, and **2.73** for **Levantine** [2508.17378].

This combination of benchmark results and UI-level evaluation suggests that ALLaM is strongest in **MSA handling**, **generation**, **code-switching**, and general Arabic assistant behavior, while **dialectal authenticity** remains materially less uniform than formal Arabic performance.

## 4. Comparative role in Arabic LLM evaluation

Later Arabic LLM studies frequently use ALLaM as a reference baseline rather than as the target of investigation. In **"Nile-Chat: Egyptian Language Models for Arabic and Latin Scripts"**, ALLaM appears as **`ALLaM-7B-Instruct-preview`** in the paper’s Arabic-script benchmark table, which covers **EgyptianMMLU**, **Belebele_Arz**, **EgyptianHellaSwag**, **EgyptianPIQA**, **EgyptianWinoGrande**, **EgyptianOpenBookQA**, **EgyptianRACE-H**, **EgyptianRACE-M**, **EgyptianAlpacaEval**, and three generation tasks: **Long Translation**, **Short Translation**, and **Transliteration** [2507.04569].

The reported ALLaM scores include **60.08** on **EgyptianMMLU**, **67.67** on **Belebele_Arz**, **62.18** on **EgyptianWinoGrande**, and **69.55** on **EgyptianAlpacaEval**. For generation, the same row reports **Long Translation** at **BLEU 26.57**, **chrF 52.59**, **BERTScore 78.34**; **Short Translation** at **BLEU 25.20**, **chrF 48.12**, **BERTScore 65.97**; and **Transliteration** at **BLEU 2.10**, **chrF 18.92**, **BERTScore 49.42**. The paper states that **Nile-Chat-12B** beats ALLaM on every listed Arabic-script benchmark and every generation metric, while the smaller Nile-Chat MoE variants exceed ALLaM on many tasks, especially translation and transliteration. At the same time, the exact table is mixed for smaller models: ALLaM remains competitive or ahead on some understanding tasks relative to **Nile-Chat-4B**, **2x4B-A6B**, and **3x4B-A6B** [2507.04569].

A key evaluation detail is scope. The Nile-Chat paper reports ALLaM only in the **Arabic-script benchmark table**, not in the **Latin-script benchmark table**. The broader narrative claim is that Nile-Chat surpasses ALLaM on Egyptian dialect understanding and generation overall, especially for **translation** and **transliteration**, but the explicit coverage for ALLaM is limited to Arabic-script evaluation [2507.04569].

This later usage is important for situating ALLaM historically: it functions simultaneously as a competitive Arabic baseline and as a model whose limitations become more visible when the benchmark stresses **dialect-specific generation** or **dual-script Egyptian Arabic** rather than primarily **MSA and English**.

## 5. Reliability, hallucination, and structured legal reasoning

ALLaM has also been studied from the standpoint of **reliability** rather than pure task accuracy. In **AraHalluEval**, ALLaM is evaluated as **Allam-preview-7b-instruct**, described there as pre-trained on **4 trillion English tokens followed by 1.2 trillion Arabic/English tokens**. The study introduces a fine-grained manual hallucination framework for Arabic **generative question answering (GQA)** and **summarization**, separating hallucination into **factuality** and **faithfulness** and using **12 fine-grained hallucination indicators in total** across the two tasks [2509.04656].

On **GQA**, ALLaM is reported as the **best-performing model** in the table, with the **lowest average hallucination score of 0.382**. Its component scores are **Named Entity 0.083**, **Value 0.240**, **Factual Contradiction 0.307**, **Conflict 0.000**, **Grammar 0.003**, **Generic 0.070**, **Knowledge Source Conflict 0.023**, **Total factuality 0.727**, **Instruction Inconsistency 0.007**, **Code-Switching 0.030**, and **Total faithfulness 0.037**. On **summarization**, the table entries are **Named Entity 0.030**, **Value 0.060**, **Fabrication 0.010**, **Inference 0.110**, **Grammar 0.000**, **Total factuality 0.210**, **Hallucination density 0.066**, **Instruction Inconsistency 0.000**, **Code-Switching 0.200**, **Total faithfulness 0.020**, and **Average 0.220**; the text then reports an overall average hallucination score of **0.215** and the **best human rating of 5**. The paper also reports a paired t-test against **Qwen2.5-7b** on summarization with **\(p = 0.0186\)**, interpreting ALLaM as significantly less hallucinatory on that task [2509.04656].

The picture is notably different in **Islamic inheritance reasoning**, where ALLaM is evaluated on **1,000 multiple-choice questions** derived from **IslamWeb fatwas** and covering heir identification, exclusion, fixed shares, residuaries, **`radd`**, **`ʿawl`**, **`munāsakha`**, and disputes such as **Akdariyya**. In that study, **ALLaM** scores **42.9% overall**, **58.0% on Beginner**, and **27.8% on Advanced**, well below **o3** at **93.4%** and **Gemini 2.5** at **90.6%** [2509.01081]. The error analysis reports **Foundational errors (FD) 45.8%** and **Complex errors (CPLX) 54.2%**, with specific categories including **Error in Normative Rules (ENR) 38.0%**, **Comprehension Error (CE) 4.2%**, **Basic Computational Error (BCE) 3.5%**, and **Error in Calculatory Adjustment (ECA) 54.2%**. Even among correct answers, **27.5%** of ALLaM’s justifications still contain foundational errors [2509.01081].

A separate QIAS 2025 systems paper reproduces the same baseline figures—**42.9% overall**, **58.0% Beginner**, **27.8% Advanced**—and shows that a domain-adapted **LoRA + RAG** system can reach **85.8% overall**, **74.0% Beginner**, and **97.6% Advanced**, far above zero-shot ALLaM [2508.15854]. This suggests that ALLaM’s Arabic-native pretraining is effective for factuality-sensitive generation, but not sufficient by itself for domains requiring tightly ordered **jurisprudential rule application** and **exact arithmetic**.

## 6. Adaptation pathways, retrieval grounding, and ambiguity of the term

A notable feature of the later literature is that ALLaM is treated as a platform for **specialization** rather than a fixed endpoint. One line of work targets dialect control. **"Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation"** fine-tunes **ALLaM-7B-Instruct-preview** on a private **Saudi Dialect Instruction dataset** of **5,466 synthetic instruction-response pairs**, balanced **50/50** between **Hijazi** and **Najdi**. Using LoRA with **rank \(r = 32\)**, **alpha \(= 64\)**, **dropout \(= 0.1\)**, **15 epochs**, and supervised causal LM training, the paper compares a **Dialect-Token** variant against a **No-Token** variant. The best model, **ALLaM-LoRA-Token**, raises the **Saudi rate** from **47.97%** to **84.21%**, reduces **MSA leakage** from **32.63%** to **6.21%**, improves **chrF++** by **+3.53**, and improves **BERTScore** by **+0.059**; it also outperforms several generic instruction models in dialect control and fidelity [2508.13525].

Another line of work uses **retrieval-augmented generation** to compensate for historical and lexical gaps. In **"Grounding Arabic LLMs in the Doha Historical Dictionary: Retrieval-Augmented Understanding of Quran and Hadith"**, ALLaM is used as an Arabic-native generator inside a DHDA-grounded RAG pipeline comprising **query preprocessing**, **intent classification**, **hybrid retrieval**, **cross-encoder re-ranking**, **intent-based routing / prompt construction**, and **LLM answer generation**. The intent classifier uses **TF-IDF features** and a **Random Forest** with **200 estimators** and confidence threshold **0.6**; retrieval uses **BM25**, dense embeddings indexed in **FAISS IndexFlatL2**, and a **BAAI/bge-reranker-v2-m3** cross-encoder fine-tuned for **3 epochs** with learning rate **\(5 \times 10^{-5}\)** under **FP16** [2603.23972].

On **1,000 meaning-based questions**, the paper reports **ALLaM baseline 56.85%** and **ALLaM + RAG 86%** in zero-shot, while the few-shot setting drops to **78%**. The authors attribute the drop to **longer prompts / longer context**. The best retrieval configurations reported are **BM25 + fine-tuned reranker** with **R@10 = 0.647**, **MRR = 0.936**, **MAP = 0.611**, and **BM25 + Nomic fusion + fine-tuned reranker** with **R@10 = 0.652**, **MRR = 0.945**, **MAP = 0.609** [2603.23972]. This suggests that ALLaM’s weaknesses on historical, diachronic, and religious Arabic are substantially reducible when the model is anchored in structured lexical evidence.

The term itself has a limited but real ambiguity. In the ALLaM literature, it denotes the Arabic-English LLM family just described. Separately, one spintronics paper notes that if “Allam” is interpreted as part of a broader **all-altermagnetic** theme, then the relevant concept is an **all-altermagnetic tunnel junction (AAMTJ)** built from **RuO\(_2\)/NiF\(_2\)/RuO\(_2\)** rather than any language model [2510.23269]. The two usages are unrelated. In present arXiv usage, however, the dominant research meaning of **Allam/ALLaM** is the Arabic-centric LLM family introduced for Arabic and English language modeling [2407.15390].

Source: https://www.emergentmind.com/topics/allam