ALLaM: Arabic-Centric Large Language Models
- ALLaM is a family of Arabic-centric transformer models designed to advance Arabic and English language technologies through specialized tokenization and bilingual pretraining.
- It employs modified tokenizers, staged pretraining on 4 trillion English tokens and 540B Arabic tokens, and fine-tuning with human preference alignment to boost learning speed.
- Benchmark results demonstrate state-of-the-art performance in MMLU Arabic and dialect evaluations, positioning ALLaM as a key baseline for Arabic LLM research.
Searching arXiv for papers on ALLaM / Allam to ground the article in current literature. Allam, in current machine-learning literature typically stylized as ALLaM, denotes a family of Arabic-centric LLMs designed for Arabic and English and positioned as infrastructure for Arabic Language Technologies (ALT). The series is presented as a response to the English-centric bias of many frontier LLMs, with an emphasis on language alignment, knowledge transfer at scale, bilingual competence, and extensive human-preference alignment. Subsequent literature treats ALLaM both as a foundation model family in its own right and as a strong Arabic-native baseline for evaluation in dialect processing, factuality, hallucination analysis, legal reasoning, and retrieval-augmented understanding of historical and religious Arabic (Bari et al., 2024).
1. Definition, naming, and model family
The name ALLaM is explicitly introduced as shorthand for Arabic LLM. In relation to the query “Allam,” the foundational paper states that the reference is to ALLaM—with capital L’s—and not to the general Arabic word “Allam” (Bari et al., 2024).
ALLaM is described as a family of autoregressive, decoder-only transformer models. The foundational release reports 7B, 13B, and 70B models initialized from Llama-2 weights, together with a 7B model trained from scratch. A later deployment-oriented evaluation describes the publicly discussed family as including 7B, 13B, 34B, and 70B parameter scales, with ALLaM-34B treated as the most capable public variant at that time and used as the backbone of HUMAIN Chat, a closed conversational web service (Bari et al., 2024, Nacar, 24 Aug 2025).
The project is framed as an Arabic-centric alternative to English-first systems. One later paper describes ALLaM-7B-Instruct-preview as the first foundation model developed in Saudi Arabia, while the UI-level study attributes the family’s introduction to the Saudi Data and AI Authority (SDAIA) (Barmandah, 19 Aug 2025, Nacar, 24 Aug 2025). Across these accounts, the consistent characterization is that ALLaM was built to improve Arabic-language understanding and generation without sacrificing strong English capability.
2. Pretraining, tokenization, and alignment strategy
The core technical narrative of ALLaM is bilingual adaptation through tokenizer expansion, staged pretraining, and alignment with human preferences. The foundational paper argues that Llama-2 tokenizes Arabic poorly, often fragmenting Arabic words into characters or byte-fallback units; to address this, the authors train an Arabic-specific tokenizer and merge it with the original Llama-2 tokenizer by adding Arabic entries not already present in the original vocabulary. They then compare two initialization schemes for the new token embeddings: random initialization versus initialization from the average of the embeddings of the token’s decomposition under the original tokenizer. The paper reports that this embedding-initialization strategy dramatically speeds up learning of Arabic (Bari et al., 2024).
Pretraining is staged around a large bilingual mixture. The paper reports about 4 trillion English tokens from sources such as Dolma-v1 and the Pile, and a curated Arabic corpus of 540B Arabic tokens total, split into 270B natural Arabic tokens and 270B translated Arabic tokens. The translated Arabic portion is important because it is used not merely as data augmentation but to improve cross-lingual knowledge alignment. The authors explicitly state that translated Arabic mitigates catastrophic forgetting in English, and they report that a roughly 45/55 Arabic/English ratio is preferred in their ablations (Bari et al., 2024).
For the from-scratch 7B model, the reported schedule is 4T English tokens followed by 1.2T mixed Arabic/English tokens. The training setup is kept close to Llama-2’s recipe, using 4M tokens per batch, a maximum learning rate of decayed to with a cosine schedule, bf16 mixed precision, Megatron-LM parallelism, FlashAttention, and an overall compute footprint estimated at 5 million GPU hours on 128–1024 A100 GPUs (Bari et al., 2024).
Alignment proceeds in two stages. Supervised fine-tuning (SFT) uses the Ultra-Instinct dataset, reported at 12M samples in v1 and evenly split between English and Arabic; v2 is smaller but cleaner. During SFT, the model is trained for 3 epochs with learning rate and batch size 1024, with prompt tokens masked in the loss. Preference alignment uses Direct Preference Optimization (DPO) rather than PPO, with batch size 512, KL penalty 0.1, learning rate from to under cosine annealing, and one epoch over preference data. The manually verified preference set contains 25,854 triplets and is expanded to 245K samples by generating multiple rejected candidates per prompt (Bari et al., 2024).
3. Benchmark performance and deployed conversational behavior
The foundational evaluation presents ALLaM as a high-performing bilingual model family. ALLaM-70B is reported to achieve state-of-the-art performance on MMLU Arabic, ACVA, and Arabic Exams, and to be the best model on five of eight Arabic benchmark sets, including both Arabic MMLU variants, Exams, ETEC, and AraTruthfulQA. On MT-Bench, ALLaM-70B Instruct scores 8.19 average in Arabic and 7.44 average in English, which the paper uses to support its claims about dialogue quality and bilingual instruction following (Bari et al., 2024).
A later deployment-oriented paper evaluates ALLaM-34B indirectly through HUMAIN Chat rather than through open weights or an API. The study constructs a prompt pack of 23 distinct prompts across seven thematic categories—MSA, Dialect, Code-switching, Knowledge, Reasoning, Generation, and Safety / security—and submits each prompt five times, yielding 115 outputs. These outputs are scored by GPT-5, Gemini 2.5 Pro, and Claude Sonnet-4 on a 1–5 Likert scale across Accuracy, Fluency, Instruction following, Safety, and Dialect fidelity (Nacar, 24 Aug 2025).
The reported category means are high: Code-switching 4.92, Generation 4.92, Knowledge 4.77, MSA 4.74, Reasoning 4.64, Safety 4.54, and Dialect 4.21. Safety subcategories are reported at 4.20 for Prompt Injection, Jailbreak, and Data Exfiltration, with effectively zero variance across runs. The same study reports response latency of roughly 1–3 seconds and interprets the adversarial safety results as stable refusal behavior. Dialect performance is more uneven: the paper reports approximate overall dialect scores of about 3.8 for Najdi, Hijazi, and Egyptian, 3.3 for Moroccan, and 2.73 for Levantine (Nacar, 24 Aug 2025).
This combination of benchmark results and UI-level evaluation suggests that ALLaM is strongest in MSA handling, generation, code-switching, and general Arabic assistant behavior, while dialectal authenticity remains materially less uniform than formal Arabic performance.
4. Comparative role in Arabic LLM evaluation
Later Arabic LLM studies frequently use ALLaM as a reference baseline rather than as the target of investigation. In "Nile-Chat: Egyptian LLMs for Arabic and Latin Scripts", ALLaM appears as ALLaM-7B-Instruct-preview in the paper’s Arabic-script benchmark table, which covers EgyptianMMLU, Belebele_Arz, EgyptianHellaSwag, EgyptianPIQA, EgyptianWinoGrande, EgyptianOpenBookQA, EgyptianRACE-H, EgyptianRACE-M, EgyptianAlpacaEval, and three generation tasks: Long Translation, Short Translation, and Transliteration (Shang et al., 6 Jul 2025).
The reported ALLaM scores include 60.08 on EgyptianMMLU, 67.67 on Belebele_Arz, 62.18 on EgyptianWinoGrande, and 69.55 on EgyptianAlpacaEval. For generation, the same row reports Long Translation at BLEU 26.57, chrF 52.59, BERTScore 78.34; Short Translation at BLEU 25.20, chrF 48.12, BERTScore 65.97; and Transliteration at BLEU 2.10, chrF 18.92, BERTScore 49.42. The paper states that Nile-Chat-12B beats ALLaM on every listed Arabic-script benchmark and every generation metric, while the smaller Nile-Chat MoE variants exceed ALLaM on many tasks, especially translation and transliteration. At the same time, the exact table is mixed for smaller models: ALLaM remains competitive or ahead on some understanding tasks relative to Nile-Chat-4B, 2x4B-A6B, and 3x4B-A6B (Shang et al., 6 Jul 2025).
A key evaluation detail is scope. The Nile-Chat paper reports ALLaM only in the Arabic-script benchmark table, not in the Latin-script benchmark table. The broader narrative claim is that Nile-Chat surpasses ALLaM on Egyptian dialect understanding and generation overall, especially for translation and transliteration, but the explicit coverage for ALLaM is limited to Arabic-script evaluation (Shang et al., 6 Jul 2025).
This later usage is important for situating ALLaM historically: it functions simultaneously as a competitive Arabic baseline and as a model whose limitations become more visible when the benchmark stresses dialect-specific generation or dual-script Egyptian Arabic rather than primarily MSA and English.
5. Reliability, hallucination, and structured legal reasoning
ALLaM has also been studied from the standpoint of reliability rather than pure task accuracy. In AraHalluEval, ALLaM is evaluated as Allam-preview-7b-instruct, described there as pre-trained on 4 trillion English tokens followed by 1.2 trillion Arabic/English tokens. The study introduces a fine-grained manual hallucination framework for Arabic generative question answering (GQA) and summarization, separating hallucination into factuality and faithfulness and using 12 fine-grained hallucination indicators in total across the two tasks (Alansari et al., 4 Sep 2025).
On GQA, ALLaM is reported as the best-performing model in the table, with the lowest average hallucination score of 0.382. Its component scores are Named Entity 0.083, Value 0.240, Factual Contradiction 0.307, Conflict 0.000, Grammar 0.003, Generic 0.070, Knowledge Source Conflict 0.023, Total factuality 0.727, Instruction Inconsistency 0.007, Code-Switching 0.030, and Total faithfulness 0.037. On summarization, the table entries are Named Entity 0.030, Value 0.060, Fabrication 0.010, Inference 0.110, Grammar 0.000, Total factuality 0.210, Hallucination density 0.066, Instruction Inconsistency 0.000, Code-Switching 0.200, Total faithfulness 0.020, and Average 0.220; the text then reports an overall average hallucination score of 0.215 and the best human rating of 5. The paper also reports a paired t-test against Qwen2.5-7b on summarization with , interpreting ALLaM as significantly less hallucinatory on that task (Alansari et al., 4 Sep 2025).
The picture is notably different in Islamic inheritance reasoning, where ALLaM is evaluated on 1,000 multiple-choice questions derived from IslamWeb fatwas and covering heir identification, exclusion, fixed shares, residuaries, radd, ʿawl, munāsakha, and disputes such as Akdariyya. In that study, ALLaM scores 42.9% overall, 58.0% on Beginner, and 27.8% on Advanced, well below o3 at 93.4% and Gemini 2.5 at 90.6% (Bouchekif et al., 1 Sep 2025). The error analysis reports Foundational errors (FD) 45.8% and Complex errors (CPLX) 54.2%, with specific categories including Error in Normative Rules (ENR) 38.0%, Comprehension Error (CE) 4.2%, Basic Computational Error (BCE) 3.5%, and Error in Calculatory Adjustment (ECA) 54.2%. Even among correct answers, 27.5% of ALLaM’s justifications still contain foundational errors (Bouchekif et al., 1 Sep 2025).
A separate QIAS 2025 systems paper reproduces the same baseline figures—42.9% overall, 58.0% Beginner, 27.8% Advanced—and shows that a domain-adapted LoRA + RAG system can reach 85.8% overall, 74.0% Beginner, and 97.6% Advanced, far above zero-shot ALLaM (AL-Smadi, 20 Aug 2025). This suggests that ALLaM’s Arabic-native pretraining is effective for factuality-sensitive generation, but not sufficient by itself for domains requiring tightly ordered jurisprudential rule application and exact arithmetic.
6. Adaptation pathways, retrieval grounding, and ambiguity of the term
A notable feature of the later literature is that ALLaM is treated as a platform for specialization rather than a fixed endpoint. One line of work targets dialect control. "Saudi-Dialect-ALLaM: LoRA Fine-Tuning for Dialectal Arabic Generation" fine-tunes ALLaM-7B-Instruct-preview on a private Saudi Dialect Instruction dataset of 5,466 synthetic instruction-response pairs, balanced 50/50 between Hijazi and Najdi. Using LoRA with rank , alpha , dropout , 15 epochs, and supervised causal LM training, the paper compares a Dialect-Token variant against a No-Token variant. The best model, ALLaM-LoRA-Token, raises the Saudi rate from 47.97% to 84.21%, reduces MSA leakage from 32.63% to 6.21%, improves chrF++ by +3.53, and improves BERTScore by +0.059; it also outperforms several generic instruction models in dialect control and fidelity (Barmandah, 19 Aug 2025).
Another line of work uses retrieval-augmented generation to compensate for historical and lexical gaps. In "Grounding Arabic LLMs in the Doha Historical Dictionary: Retrieval-Augmented Understanding of Quran and Hadith", ALLaM is used as an Arabic-native generator inside a DHDA-grounded RAG pipeline comprising query preprocessing, intent classification, hybrid retrieval, cross-encoder re-ranking, intent-based routing / prompt construction, and LLM answer generation. The intent classifier uses TF-IDF features and a Random Forest with 200 estimators and confidence threshold 0.6; retrieval uses BM25, dense embeddings indexed in FAISS IndexFlatL2, and a BAAI/bge-reranker-v2-m3 cross-encoder fine-tuned for 3 epochs with learning rate under FP16 (Eltanbouly et al., 25 Mar 2026).
On 1,000 meaning-based questions, the paper reports ALLaM baseline 56.85% and ALLaM + RAG 86% in zero-shot, while the few-shot setting drops to 78%. The authors attribute the drop to longer prompts / longer context. The best retrieval configurations reported are BM25 + fine-tuned reranker with R@10 = 0.647, MRR = 0.936, MAP = 0.611, and BM25 + Nomic fusion + fine-tuned reranker with R@10 = 0.652, MRR = 0.945, MAP = 0.609 (Eltanbouly et al., 25 Mar 2026). This suggests that ALLaM’s weaknesses on historical, diachronic, and religious Arabic are substantially reducible when the model is anchored in structured lexical evidence.
The term itself has a limited but real ambiguity. In the ALLaM literature, it denotes the Arabic-English LLM family just described. Separately, one spintronics paper notes that if “Allam” is interpreted as part of a broader all-altermagnetic theme, then the relevant concept is an all-altermagnetic tunnel junction (AAMTJ) built from RuO0/NiF1/RuO2 rather than any LLM (Zhang et al., 27 Oct 2025). The two usages are unrelated. In present arXiv usage, however, the dominant research meaning of Allam/ALLaM is the Arabic-centric LLM family introduced for Arabic and English language modeling (Bari et al., 2024).
