--- title: Arabic Text-Grounded SFT Reasoning Pairs url: https://www.emergentmind.com/topics/arabic-text-grounded-sft-reasoning-pairs type: topic --- # Arabic Text-Grounded SFT Reasoning Pairs Arabic Text-Grounded SFT Reasoning Pairs are large-scale collections of supervised fine-tuning instances in which Arabic-language questions, instructions, or prompts are paired with one or more grounded source passages (such as text snippets, tables, or images), a structured chain-of-thought (CoT) style reasoning sequence, and a final answer or label. These corpora enable the fine-tuning and evaluation of large language models (LLMs) and multimodal models on Arabic stepwise reasoning, evidence tracing, and culturally situated question answering. Recent benchmark datasets span Islamic law, multimodal visual/textual reasoning, and dialectal cultural knowledge, establishing rigorous frameworks for representation, annotation, and downstream performance analysis. ## 1. Corpus Construction and Data Sources Text-grounded SFT reasoning pairs for Arabic originate from diverse, high-quality sources, each selected and segmented to support instructive, evidence-based question answering. - **Religious Texts and Legal Sources**: The ISLAMICFAITHQA suite compiles 25,000 Arabic SFT pairs grounded in (i) Qur’ān verses (6,236 ayat, each indexed by surah/ayah), (ii) canonical hadīth collections (with isnād and text), and (iii) major tafsīr commentaries to support multi-step reasoning about Islamic law, ritual, and doctrine. Candidate passages are extracted with domain-relevant keyword filters and expert curation, prioritizing items tagged as “hard” or “very hard” to ensure a reasoning-intensive distribution. Three subject-area specialists review all passages for doctrinal validity [2601.07528]. - **Multimodal and Domain-Varied Benchmarks**: The ARB corpus collects 1,356 multimodal samples (documents, OCR scans, images, tables) spanning 11 domains—of which the text-only domains (Document Understanding, OCR) yield pure text-grounded reasoning chains. Textual inputs are drawn from translated/localized English benchmarks, Arabic QA tasks (CAMEL-Bench), synthetic LLM generations, and tool-assisted creation for tabular/diagrammatic input [2505.17021]. - **Cultural Question-Answering and Dialectal Variety**: Open-ended Arabic cultural QA datasets begin from Modern Standard Arabic (MSA) MCQs (e.g., PalmX-GC, 2,000+ entries), producing parallel SFT pairs across five major Arabic varieties (MSA, Egyptian, Levantine, Gulf, Maghrebi) and English. Dialectal translations and open-ended conversions are performed by GPT-4.1 under controlled prompts with semantic equivalence, followed by QA chain-of-thought rationale generation and verification [2510.24328]. - **Natural Language Inference**: The Arabic NLI SFT sets compile premise–hypothesis–label triples from XNLI-Arabic, SNLI (translated), and machine-translated arNLI, yielding balanced splits for entailment, contradiction, and neutral relations (total: 14,758 examples) [2307.14666]. ## 2. Annotation, Chain-of-Thought Generation, and Quality Assurance Annotation and verification protocols are engineered to maximize the faithfulness, cultural alignment, and logical completeness of each SFT pair. - **LLM-Assisted Generation and Human Adjudication:** For Islamic jurisprudence and Creed SFT pairs, Arabic-fluent LLMs are directed by CoT-specific prompts to generate (a) a well-grounded question, (b) an explicit stepwise reasoning chain citing specific texts (e.g., Qur’ān, hadīth), and (c) a final canonical answer. Each example is cross-checked by three domain experts, with subsample validation assessments and Cohen’s κ measuring inter-annotator reliability (κ = 0.78 for reasoning, 0.72 for answers) [2601.07528]. - **Multiphase Curation and Verification:** In ARB, each multimodal sample receives two rounds of review by native Arabic annotators, jointly scoring grammatical fluency, logical coherence, faithfulness, “commonsense” reasoning, and cultural grounding. Items below a quality threshold are redrafted or removed. Krippendorff’s α = 0.8356 for human agreement, increasing to 0.8762 with GPT-4o as a judge [2505.17021]. - **Dialectal and Cultural Transfer:** For “Beyond MCQ”, MCQ–OEQ conversion and dialectal adaptation pass through controlled LLM prompts and human review (Likert ≈ 4.4/5 for translation adequacy). Chain-of-thought rationales are then sampled, filtered, and scored for gold answer alignment (automated match Jaccard ≥ 0.75, confidence threshold σ ≥ 0.8), with only verified instances included in the SFT release [2510.24328]. ## 3. Schema, Format, and Logical Form Each SFT reasoning pair is presented in a structured JSON schema designed for ingestion by LLM SFT pipelines and to maximize traceability between input, reasoning, evidence, and output. | Field | Description | Example | |---------------|----------------------------------------|------------------------------------------------------------------------------------------------------------------------| | instruction | User prompt/question | "متى يبدأ وقت صلاة الفجر؟" | | context | Grounded text (ayah/hadith/table/etc.) | "آية 2:187: ... وَأَقِمِ الصَّلَاةَ لِدُلُوكِ الشَّمْسِ ..." | | reasoning | Ordered steps, often with citations | ["1) بداية الفجر الصادق.", "2) الآية 2:187 تدل على ذلك.", "3) العلماء اتفقوا على هذا التوقيت."] | | answer | Final claim/label | "وقت بداية صلاة الفجر هو عند الفجر الصادق." | - In ARB, an analogous format is used: “prompt”, ordered “steps” (each with a discrete “action” label such as “identify_text”, “arithmetic”, “interpret”), and “answer". No fixed step count exists, but reasoning chains span 2–6 steps in most cases [2505.17021]. - For NLI, triples are recast as instruction–input–output SFT pairs. The template is: - Instruction: “حددُ العلاقةَ المنطقية بين الجملتين الآتيتين.” - Input: Premise: “…”, Hypothesis: “…” - Output: “تضمن” | “محايد” | “تناقض" [2307.14666]. - Dialectal SFT pairs for cultural QA are defined as tuples \((x_i,\,r_i,\,y_i)\), where \(x_i\) is the open-ended question, \(r_i\) the CoT rationale, and \(y_i\) the gold answer, with parallel realizations in MSA and four dialects [2510.24328]. - Complex logical forms often emerge, especially in religious domains: Premise(n) (text citation), Inference(n), then Conclusion, employing explicit connective structures (“إذ”، “لأنه”) [2601.07528]. ## 4. Domain Coverage, Statistics, and Reasoning Complexity Arabic text-grounded SFT datasets now encompass an array of domains, linguistic varieties, and reasoning complexities. - **Islamic QA**: 78% of ISLAMICFAITHQA SFTs are on Worship & Fiqh, 12% Creed & Morality, 10% Qur’ān/Hadith Studies. Reasoning chains average 3.8 steps (45% 2–3 steps, 40% 4–5 steps, 15% 6–7 steps) [2601.07528]. - **ARB Domains**: 1,356 samples, 5,119 step-action pairs, 11 domains (text-only Document Understanding and OCR yield pure text chains). Mean reasoning depth ≈3.78 steps (per sample), with varied task-relevant actions [2505.17021]. - **Cultural QA and Dialectal Coverage**: 2,000 SFT pairs per dialect (MSA, Egyptian, Levantine, Gulf, Maghrebi), plus English, for a total of 10,000. Average CoT rationale ≈45 tokens (200 chars), topics uniformly distributed across history, geography, arts, cuisine, customs, and notable figures [2510.24328]. - **NLI**: 14,758 triples; label splits near 1/3 for each of entailment, contradiction, and neutral. Used for both reasoning classification and as edge-case SFTs for fine-tuning [2307.14666]. ## 5. Evaluation, Metrics, and Impact on Model Behavior Text-grounded Arabic SFT pairs provide detailed frameworks for downstream evaluation, interpretability, and improvement of LLM reasoning. - **Quality Metrics**: ARB scores each reasoning chain along five dimensions (Faithfulness, Informativeness, Coherence, Commonsense, Reasoning Alignment), with the final reasoning score given by: $$ S = \frac{1}{K}\sum_{i=1}^{K} s_i $$ where \(K=5\). Inter-annotator agreement scores (Krippendorff’s α) reach 0.8356 (human only), up to 0.8762 with GPT-4o inclusion [2505.17021]. - **Supervised Fine-Tuning (SFT) Effect**: In ISLAMICFAITHQA, fine-tuning on the 25,000 SFT pairs increases accuracy on QuranicQA by ΔAcc = +1.05 for a 9B parameter model (Fanar-1), and ΔAcc = +8.70 for Qwen3-4B [2601.07528]. Downstream, agentic tool-driven retrieval (agentic RAG) further boosts correctness and sharply reduces both ungrounded answers and unwarranted attempts. - **NLI Evaluation**: AraBERT and XLM-RoBERTa models fine-tuned on SFT NLI pairs achieve test set accuracy/F1—AraBERT baseline: 75.3/75.4%; XLM-R (baseline): 78.7/78.8%; with multi-task NER improvement up to 88.1% for contradiction detection. Contradiction detection particularly benefits from explicit entity (person/place) recognition and numerical clash handling. CoT rationales are used for edge cases (negation, quantifier scope, temporal shifts) [2307.14666]. - **Dialectal CoT Benchmarking**: On “Beyond MCQ”, CoT fine-tuning slightly raises judged correctness scores (ΔJ ≈ +0.14), while token-level n-gram metrics may decrease (reflecting shorter yet more grounded rationales). Multi-dialectal QA highlights persistent LLM competence gaps on dialect and cultural knowledge [2510.24328]. ## 6. Representative Examples The following examples illustrate typical SFT input–reasoning–output structures reported in the literature: **Islamic Law, Worship:** ```json { "instruction": "متى يبدأ وقت صلاة الفجر؟", "context": "آية 2:187: ...", "question": "متى يبدأ وقت صلاة الفجر؟", "reasoning": [ "1) بداية الفجر الصادق.", "2) الآية 2:187 تدل على ذلك.", "3) العلماء اتفقوا على هذا التوقيت." ], "answer": "وقت بداية صلاة الفجر هو عند الفجر الصادق." } ``` **Document Understanding (Sales Table):** ```json { "prompt": "في الجدول التالي، ما مجموع مبيعات يناير وفبراير؟", "steps": [ {"step": "أنظر إلى عمود 'يناير' وأجد القيمة ٥٠٠.", "action": "read_cell"}, {"step": "أنظر إلى عمود 'فبراير' وأجد القيمة ٤٠٠.", "action": "read_cell"}, {"step": "أجمع ٥٠٠ + ٤٠٠ = ٩٠٠.", "action": "arithmetic"} ], "answer": "٩٠٠" } ``` **Cultural QA (MSA; Capital City):** \[ x = \text{“ما هي عاصمة ليبيا؟”},\quad r = \text{“أولًا، تقع ليبيا في شمال أفريقيا على البحر المتوسط... طرابلس هي الموقع الأقدم والأكبر، وعادة ما تكون العاصمة.”},\quad y = \text{“طرابلس”}. \] ## 7. Distinctive Features and Research Significance Arabic text-grounded SFT reasoning pairs enable high-fidelity, stepwise modeling in domains that historically lack robust supervised corpora. Their design emphasizes: - **Grounding and Transparency**: Each answer is explicitly tied to verifiable source passages, structured for both model tracing and error analysis (notably for reducing hallucination and enforcing abstention when evidence is lacking) [2601.07528]. - **Cultural and Dialectal Breadth**: Parallel SFT datasets in multiple dialects and knowledge categories facilitate fine-grained cultural adaptation and expose persistent model weaknesses on Arabic linguistic phenomena [2510.24328]. - **Evaluation Rigor**: Metrics such as CoT coherence, faithfulness, and human/COT inter-annotator agreement set rigorous standards for claim verification, reasoning completeness, and model robustness [2505.17021]. - **Workflow Replicability**: Documented sampling, prompting, annotation, and schema protocols allow reproducibility and modular extensibility to new domains or language varieties. A plausible implication is that such SFT collections enable end-to-end Arabic LLMs with explainable, faithful, and culturally aligned behaviors, positioning them for deployment in high-stakes domains (law, religion, education, government) where ungrounded outputs are unacceptable. Source: https://www.emergentmind.com/topics/arabic-text-grounded-sft-reasoning-pairs