---
title: Shakkala Neural System for Arabic Diacritization
url: https://www.emergentmind.com/topics/shakkala-neural-system
type: topic
---

# Shakkala Neural System for Arabic Diacritization

The Shakkala Neural System is an end-to-end deep learning framework for automatic Arabic text diacritization, leveraging character-level bidirectional long short-term memory (Bi-LSTM) networks with learned embeddings. Introduced by Barqawi & Zerrouki, Shakkala addresses the insertion of diacritics (short vowels and other phonetic marks) in Arabic texts—a challenge essential for downstream tasks such as text-to-speech synthesis and language learning—by modeling fine-grained orthographic and contextual dependencies at the character level. On a rigorously cleaned benchmark derived from the Tashkeela Corpus, Shakkala achieves a Diacritic Error Rate (DER) of 2.88%, significantly outperforming prior rule-based and statistical approaches [1905.01965].

## 1. System Architecture and Inference Pipeline

Shakkala operates as a character-level sequence labeling model. At inference, input text is segmented into blocks of up to 315 characters (to accommodate model constraints). Each segment undergoes the following processing pipeline:

- **Encoding:** Each character is mapped to an integer index over a vocabulary of approximately 60–70 Arabic characters, punctuations, and a dedicated “unknown” (UNK) token.
- **Embedding:** Characters are embedded via a trainable matrix $E \in \mathbb{R}^{|V| \times d_c}$, producing dense vectors $x_t \in \mathbb{R}^{d_c}$.
- **Sequence Encoding:** Two LSTMs operate bidirectionally:
  - The forward LSTM processes the sequence from $x_1$ to $x_T$,
  - The backward LSTM from $x_T$ to $x_1$,
  - Their hidden states at each time step are concatenated: $h_t = [h^f_t; h^b_t] \in \mathbb{R}^{2d_h}$.
- **Prediction:** Each concatenated state $h_t$ is projected to a diacritic label distribution via a linear transformation and softmax:
  $$
  z_t = W_o h_t + b_o,\quad p(y_t = k | c_{1:T}) = \text{softmax}(z_t)_k
  $$
  The predicted diacritic label is $\hat{y}_t = \arg\max_{k \in D} p(y_t = k | c_{1:T})$.

A schematic overview:
```
[c₁,...,c_T] → Embedding (E) → [x₁,...,x_T] → Bi-LSTM → [h₁,...,h_T] → Dense + Softmax → [ŷ₁,...,ŷ_T]
```
Input constraints are handled by segmenting input strings; concatenation of model outputs reconstitutes the fully diacritized line.

## 2. Mathematical Formulation

Let $c_{1:T}$ denote a sequence over the character vocabulary $V$, with a target diacritic label set $D$ ($K \approx 9$, including “no-diacritic”). The system’s computation deconstructs as:

- **Embedding:** $x_t = E \cdot \text{onehot}(c_t)$
- **Bidirectional LSTM:**
  $$
  h^f_t = \text{LSTM}_f(x_t, h^f_{t-1}),\quad h^b_t = \text{LSTM}_b(x_t, h^b_{t+1}),\quad h_t = [h^f_t; h^b_t]
  $$
- **Output & Prediction:**
  $$
  z_t = W_o h_t + b_o,\quad p(y_t = k | c_{1:T}) = \frac{e^{z_{t,k}}}{\sum_{j=1}^K e^{z_{t,j}}}
  $$
  $\hat{y}_t = \arg\max_{k \in D} p(y_t = k | c_{1:T})$

- **Training Objective:** The cross-entropy loss over $N$ training sequences:
  $$
  \mathcal{L} = -\frac{1}{N}\sum_{i=1}^N\sum_{t=1}^{T_i}\sum_{k=1}^K \mathbf{1}[y_{i,t}=k]\log p(y_{i,t}=k|c^{(i)}_{1:T_i})
  $$
- **Evaluation Metric:** Diacritic Error Rate (DER):
  $$
  \text{DER} = \frac{\# \text{diacritic errors}}{\text{total diacritics}} \times 100\%
  $$

## 3. Training Data Construction and Preprocessing

Shakkala’s benchmarking relies on a carefully constructed corpus. The procedure is as follows:

- **Source material:** Aggregation of 97 Classical Arabic books and 293 Modern Standard Arabic (MSA) documents from the Tashkeela Corpus, with additional inclusion of a simplified Qur’an version.
- **Cleaning:** Automated scripts remove HTML tags, URLs, misplaced/separated diacritics (using regular expressions for specific cases such as ending diacritics and *Fathatan + Alif* corrections), non-Arabic/Kashida characters, and extraneous whitespace. Numbers are separated from letters, and spaces are collapsed.
- **Dataset Statistics:** The resulting corpus consists of 55,000 lines (≈2.3M words). Only lines with at least 80% diacritized characters are retained.
- **Corpus Split:**

  | Subset     | Lines  | Words    | Avg. Words/Line | 
  |------------|--------|----------|-----------------|
  | Training   | 50,000 | 2.10M    | 42              |
  | Validation | 2,500  | 102K     | 42              |
  | Test       | 2,500  | 107K     | 42              |

- **Tokenization:** Strictly character-level; the only out-of-vocabulary scenario involves rare characters assigned to the UNK token.

## 4. Model Hyperparameters and Optimization

The primary source does not disclose exhaustive hyperparameter settings, but indicative values from the authors’ codebase are:

- Character embedding dimension: $d_c \approx 100$
- LSTM hidden size: $d_h \approx 200$ per direction (2 stacked layers)
- Dropout: 50% between LSTM layers
- Optimizer: Adam, learning rate $\approx 1 \times 10^{-3}$
- Batch size: 64 sequences
- Training duration: 10–20 epochs using early stopping determined by validation DER

The character limit per input (315 characters) is enforced throughout inference.

## 5. Empirical Performance and Error Analysis

Shakkala is evaluated using two metrics: Diacritic Error Rate (DER) and Word Error Rate (WER), under stringent settings (“no-diacritic” included and “without case ending”). Performance comparison with competing systems is summarized below:

| System           | DER (%) | WER (%) |
|------------------|---------|---------|
| Farasa           | 23.93   | 53.13   |
| Harakat          | 17.03   | 32.03   |
| MADAMIRA         | 29.94   | 59.07   |
| Mishkal          | 13.78   | 26.42   |
| Tashkeela-Model  | 52.96   | 94.16   |
| **Shakkala**     | **2.88**| **6.53**|

- **Error Analysis:** Shakkala exhibits substantial accuracy gains, particularly in final-letter diacritic prediction—a well-established challenge in Arabic NLP. Persisting errors are predominantly observed in rare morphological contexts and with loanwords not well represented in the training set.
- **Ablation:** No ablation studies are reported.

## 6. Contributing Factors, Limitations, and Open Challenges

- **Key Enablers:**
  - Character-level embeddings obviate dependence on manual feature engineering, facilitating efficient learning of orthographic/diacritic regularities.
  - Bidirectional LSTMs enable holistic, long-range modeling, crucial for the resolution of syntax- or morphology-dependent diacritic ambiguity.
  - The scale and cleanliness of the Tashkeela-derived corpus (≈2.1M words) support robust statistical learning.

- **Constraints:**
  - Hard input segment limit of 315 characters per inference pass (necessitating text segmentation).
  - Lack of a formal API; interaction is via a web interface or direct code inspection.
  - No performance validation on colloquial Arabic or texts outside Classical/MSA genres.
  - Error concentration in low-frequency morphological formations suggests a data-driven limitation.

A plausible implication is that further gains may require either corpus expansion with greater genre diversity or architectural augmentation for better handling of data sparsity [1905.01965].

Source: https://www.emergentmind.com/topics/shakkala-neural-system