---
title: 'AraDhati+: Arabic Subjectivity Dataset'
url: https://www.emergentmind.com/topics/aradhati
type: topic
---

# AraDhati+: Arabic Subjectivity Dataset

Searching arXiv for the specified paper to ground the article.
AraDhati+ is a curated Arabic subjectivity classification dataset introduced in “Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation” [2508.19966]. It is designed to address the shortage of annotated resources for deciding whether a text is subjective—expressing opinions, feelings, or attitudes—or objective, stating verifiable facts. The dataset is assembled by pooling four public Arabic corpora, ASTD, LABR, HARD, and SANAD, and by applying domain-informed label-mapping heuristics that collapse heterogeneous source annotations into a binary task: subjective (label 1) versus objective (label 0). In the associated framework, AraDhati+ also serves as the basis for fine-tuning XLM-RoBERTa, AraBERT, and ArabianGPT, as well as for a voting-based ensemble that attains 97.79% accuracy on the augmented test split [2508.19966].

## 1. Conceptual definition and task formulation

AraDhati+ is centered on binary subjectivity classification rather than sentiment polarity classification. In the paper’s formulation, subjectivity denotes any text segment whose truth value depends on personal stance or evaluative language, whereas objectivity denotes statements grounded in facts independent of the author’s viewpoint [2508.19966]. This distinction is important because much of the available Arabic NLP data is sentiment-focused rather than explicitly subjectivity-focused.

The dataset therefore operationalizes subjectivity through heuristic relabeling of existing resources. Tweets explicitly annotated as “subjective” in ASTD remain subjective. In review datasets, all sentiment-bearing texts, including positive, negative, and neutral reviews, are mapped to the subjective class. Objective material is sourced from selected SANAD news categories—Medical, Sports, and Technology—chosen to maximize factual reporting and minimize opinionated discourse [2508.19966]. For convenience, original sentiment tags such as POS, NEG, and NEUTRAL are retained when present, but they collapse into the single subjective super-class for the downstream task.

A common misconception is to treat neutrality in review corpora as equivalent to objectivity. AraDhati+ explicitly rejects that equivalence: “neutral” in review contexts is still treated as subjective because it reflects stance, even when the stance is ambivalent [2508.19966].

## 2. Source corpora and construction heuristics

AraDhati+ pools four public Arabic corpora spanning multiple domains and language varieties. ASTD contributes tweets in MSA and dialectal Arabic. LABR contributes book reviews with 1–5 ratings. HARD contributes hotel reviews in MSA and dialect. SANAD contributes news articles from seven categories, of which only Medical, Sports, and Technology are used in AraDhati+ as objective exemplars [2508.19966].

| Source corpus | Original annotation | AraDhati+ role |
|---|---|---|
| ASTD | Objective vs Subjective (Positive/Negative/Mixed) | Mixed source of subjective and objective |
| LABR | Ratings mapped to positive, negative, neutral | All included instances mapped to subjective |
| HARD | Ratings mapped to negative and positive; rating 3 removed in balanced version | All included instances mapped to subjective |
| SANAD | Single-labeled news categories | Medical, Sports, Technology mapped to objective |

The original ASTD distribution is 3,315 subjective and 6,691 objective, with approximately 10,006 tweets, a maximum of 45 tokens per tweet, an average of 16, 160,206 total tokens, and a vocabulary of 38,743 [2508.19966]. LABR contains more than 63,000 reviews. HARD uses a balanced subset of 94,052 reviews. SANAD contains 194,797 articles across seven categories, from which AraDhati+ selects 32,500 objective articles [2508.19966].

The construction strategy combines direct reuse and heuristic abstraction. Subjective examples consist of ASTD subjective labels together with all LABR and HARD reviews labeled positive, negative, or neutral. Objective examples consist of the selected SANAD texts. No manual re-annotation is reported; the labels are mapped automatically through clear, domain-informed heuristics [2508.19966]. This suggests that AraDhati+ is best understood as a high-coverage, heuristically labeled resource rather than a manually adjudicated gold-standard benchmark.

## 3. Balancing strategy, composition, and coverage

To counter ASTD’s imbalance and enlarge domain coverage, the dataset construction uses both oversampling and augmentation. The authors oversample ASTD subjective instances to construct balanced ASTD-only training for some model variants. They also augment the corpus with 32,500 SANAD objective articles and 32,500 subjective reviews equally drawn from LABR and HARD, 16,250 each, before including ASTD to form AraDhati+ [2508.19966].

The final AraDhati+ dataset contains 77,916 instances, split into 62,332 training instances and 15,584 test instances under an 80/20 partition [2508.19966]. The per-source train/test breakdown is 10,332/2,584 for ASTD, 13,000/3,250 for LABR, 13,000/3,250 for HARD, and 26,000/6,500 for SANAD. The class distribution is constructed to be near-balanced overall via the 32,500 subjective versus 32,500 objective addition, while ASTD contributes mixed labels and makes the overall balance slightly objective-leaning relative to the exact 32.5k/32.5k core [2508.19966].

AraDhati+ covers both MSA and dialectal Arabic, especially through HARD and ASTD, and spans tweets, book reviews, hotel reviews, and news. The dataset may contain occasional English words in tweets, although the extent of code-switching is not quantified. The paper notes that multilingual pretraining of XLM-R, without language embeddings, is beneficial for code-switching [2508.19966]. A plausible implication is that the dataset’s cross-domain construction was intended not only to enlarge sample size but also to expose models to heterogeneous discourse structures and register variation.

## 4. Preprocessing, representation, and released schema

The preprocessing pipeline applies cleaning and normalization before tokenization. The reported cleaning removes URLs, non-Arabic characters, punctuation marks, special characters, single-letter tokens, and other non-useful text artifacts [2508.19966]. Lexical normalization is applied where applicable, and stopword removal is explicitly avoided in order to preserve cues important for subjectivity and sentiment. Model-specific tokenizers are then used with truncation and padding to a maximum sequence length of 256.

The tokenization stack depends on the model family: XLMRobertaTokenizerFast for XLM-RoBERTa, BertTokenizerFast for AraBERT, and AraNizer for ArabianGPT [2508.19966]. Heuristics for social-media-specific tokens such as mentions, emojis, and hashtags are implicitly covered by cleaning and normalization, while links are stripped. Diacritics handling and specific Alef, Yaa, and Taa Marbuta normalization are not detailed.

The released CSV is structured around the fields Text, Class, Domain, Label, and Dataset. Text stores the raw Arabic string. Class stores the sentiment or tag, including POS, NEG, NEUTRAL for reviews and OBJ for objective news. Domain distinguishes Tweets, Book reviews, Hotel reviews, and Sports, Medical, or Technology news. Label stores the binary subjectivity target, and Dataset identifies the source corpus, ASTD, LABR, HARD, or SANAD [2508.19966].

The repository is available at `https://github.com/Attia14/AraDhati`. The source datasets are free and publicly accessible under their own terms, but the explicit license for AraDhati+ itself is not specified in the paper, and users are directed to consult the repository and respect the source dataset licenses [2508.19966].

## 5. Fine-tuned models and optimization regime

The paper fine-tunes three pre-trained language models in two scenarios: on oversampled ASTD only, denoted by `_1` variants, and on the augmented AraDhati+ corpus, denoted by `_2` variants [2508.19966]. XLM-RoBERTa is described as multilingual and pre-trained with masked language modeling on 2.5 TB of CommonCrawl. AraBERT is specified as an Arabic BERT with 12 layers, 768 hidden units, 12 attention heads, maximum length 512, and approximately 136M parameters. ArabianGPT-01B is a GPT-2 architecture adapted to Arabic with 12 decoder layers, approximately 134M parameters, and training on approximately 15.5GB of Arabic news [2508.19966].

These yield six fine-tuned systems: AraSubjXLM-R_1, AraSubjXLM-R_2, AraSubjBERT_1, AraSubjBERT_2, AraSubjGPT_1, and AraSubjGPT_2. Training uses AdamW with mini-batch gradient descent, batch size 16, learning rates explored over \(\{5\mathrm{e}{-6}, 1.5\mathrm{e}{-5}, 2.0\mathrm{e}{-5}, 5\mathrm{e}{-5}\}\), and epochs in \(\{1, 2, 3, 5, 7\}\) [2508.19966]. The hardware is a GPU platform, with no further specification.

The loss is standard cross-entropy for two-class classification:

$$
\mathcal{L} = -\sum_{n} \sum_{c \in \{0,1\}} y_{n,c} \log p_{n,c}
$$

The formulation situates AraDhati+ squarely within a conventional Transformer fine-tuning regime rather than a task-specific architectural redesign. This suggests that the primary research contribution lies in dataset construction and cross-domain subjectivity framing, with model diversity and ensembling used to probe robustness across domains [2508.19966].

## 6. Evaluation protocol, ensemble design, and empirical results

Evaluation uses a hold-out protocol with a stratified 20% test split per source. The paper also includes cross-domain generalization checks by evaluating models on subjective-only partitions, formed from LABR and HARD, and objective-only partitions, formed from SANAD [2508.19966]. Reported metrics are accuracy, precision, recall, and F1-score:

$$
\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}
$$

$$
\text{Precision} = \frac{TP}{TP + FP}
$$

$$
\text{Recall} = \frac{TP}{TP + FN}
$$

$$
F1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}
$$

Per-class breakdowns, confusion matrices, and confidence intervals are not provided [2508.19966].

Two ensembles are defined by voting over the three fine-tuned base models. Decision_1 combines the `_1` variants trained on oversampled ASTD. Decision_2 combines the `_2` variants trained on AraDhati+ [2508.19966]. The implementation is described as majority voting over predicted labels. The paper also gives a probability-aggregation formulation,

$$
s_c = \sum_i w_i \, p_i(c), \qquad \hat{y} = \arg\max_c s_c
$$

with uniform weights \(w_i = 1\), although majority voting is the reported method and no learned weights are given.

The headline result is that Decision_2 achieves 97.79% accuracy on the augmented AraDhati+ test set [2508.19966]. Model-wise, under oversampled ASTD training, AraSubjGPT_1 achieves the best accuracy on ASTD’s own test split at 87.78%; AraSubjXLM-R_1 obtains the best generalization to objective-only SANAD at approximately 98% accuracy; and AraSubjBERT_1 is strongest on subjective reviews, LABR plus HARD, at approximately 82% accuracy. Decision_1 reaches 95.62% on the ASTD test, 88.60% on the augmented test, and competitive performance on subjective-only and objective-only partitions at approximately 80.03% and approximately 94.37%, respectively [2508.19966].

Under augmented AraDhati+ training, AraSubjGPT_2 and AraSubjBERT_2 obtain top accuracies on the ASTD test at approximately 86%, while all three `_2` models exceed 99% accuracy on objective SANAD and subjective LABR plus HARD partitions [2508.19966]. Decision_2 attains 87.27% on the ASTD test and 97.79% on the augmented AraDhati+ test. The paper attributes the slight reduction on the original ASTD test, between \(-1.28\%\) and \(-1.78\%\) across models, to domain shift: the augmented training distribution includes reviews and news, which introduce cues less aligned with tweet subjectivity [2508.19966].

## 7. Error profile, limitations, and research significance

The error analysis for Decision_2 identifies systematic failure modes. Of all errors, 30.40% are cases in which all three component models misclassify the same instance, indicating shared limitations rather than isolated model-specific failures [2508.19966]. Within that subset, three categories are reported: Mixed Tweets at 41%, Model errors at 33%, and Short Tweets at 26%.

Mixed Tweets combine factual clauses with opinionated segments, making binary sentence-level subjectivity difficult when factual scaffolding and evaluative content co-occur. Model errors include persuasive or news-like subjective posts incorrectly flagged as objective, as well as emotive short statements that are misread. Short Tweets provide too little context for reliable classification [2508.19966]. The paper further highlights Arabic-specific challenges, including rich morphology, affixes, clitics, dialect variation, orthographic variability, sarcasm, irony, and figurative language, especially in microtext with code-switching and hashtags.

The dataset limitations are stated explicitly: automatic label mapping, the treatment of neutral reviews as subjective, no human verification for the SANAD objectivity slice, potential residual imbalance due to ASTD inclusion, domain skew across news, reviews, and tweets, and the absence of inter-annotator agreement [2508.19966]. Quality-control limitations are also noted: deduplication and extra filtering beyond the stated cleaning are not documented. On the modeling side, the paper identifies sensitivity to domain shift, brittle behavior on short and noisy social content, and limited handling of mixed-content sentences.

Future directions include expanding sources to forums, encyclopedic content, and multi-genre corpora; adding manual annotations and adjudication for a gold-standard subjectivity dataset; introducing clause-level and aspect-level subjectivity; pursuing cross-lingual transfer and domain adaptation; adopting error-aware training through curriculum learning and contrastive objectives; and improving handling of irony and sarcasm [2508.19966]. Within Arabic NLP, AraDhati+ is therefore significant less as a closed benchmark than as a multi-domain resource that exposes the distinction between sentiment and subjectivity, and that makes domain-informed heuristic relabeling a concrete basis for large-scale Arabic subjectivity evaluation.

Source: https://www.emergentmind.com/topics/aradhati