AraDhati+: Arabic Subjectivity Dataset

• AraDhati+ is a binary Arabic subjectivity classification dataset that distinguishes subjective texts (expressing opinions) from objective texts (stating facts).
• It is constructed by pooling four public corpora (ASTD, LABR, HARD, SANAD) and applying domain-informed heuristics to relabel diverse annotations into a unified binary format.
• Fine-tuned models using AraDhati+ achieve high performance—up to 97.79% accuracy—with ensemble methods and tailored preprocessing techniques.

Searching arXiv for the specified paper to ground the article. AraDhati+ is a curated Arabic subjectivity classification dataset introduced in “Dhati+: Fine-tuned LLMs for Arabic Subjectivity Evaluation” (Bellaouar et al., 27 Aug 2025). It is designed to address the shortage of annotated resources for deciding whether a text is subjective—expressing opinions, feelings, or attitudes—or objective, stating verifiable facts. The dataset is assembled by pooling four public Arabic corpora, ASTD, LABR, HARD, and SANAD, and by applying domain-informed label-mapping heuristics that collapse heterogeneous source annotations into a binary task: subjective (label 1) versus objective (label 0). In the associated framework, AraDhati+ also serves as the basis for fine-tuning XLM-RoBERTa, AraBERT, and ArabianGPT, as well as for a voting-based ensemble that attains 97.79% accuracy on the augmented test split (Bellaouar et al., 27 Aug 2025).

1. Conceptual definition and task formulation

AraDhati+ is centered on binary subjectivity classification rather than sentiment polarity classification. In the paper’s formulation, subjectivity denotes any text segment whose truth value depends on personal stance or evaluative language, whereas objectivity denotes statements grounded in facts independent of the author’s viewpoint (Bellaouar et al., 27 Aug 2025). This distinction is important because much of the available Arabic NLP data is sentiment-focused rather than explicitly subjectivity-focused.

The dataset therefore operationalizes subjectivity through heuristic relabeling of existing resources. Tweets explicitly annotated as “subjective” in ASTD remain subjective. In review datasets, all sentiment-bearing texts, including positive, negative, and neutral reviews, are mapped to the subjective class. Objective material is sourced from selected SANAD news categories—Medical, Sports, and Technology—chosen to maximize factual reporting and minimize opinionated discourse (Bellaouar et al., 27 Aug 2025). For convenience, original sentiment tags such as POS, NEG, and NEUTRAL are retained when present, but they collapse into the single subjective super-class for the downstream task.

A common misconception is to treat neutrality in review corpora as equivalent to objectivity. AraDhati+ explicitly rejects that equivalence: “neutral” in review contexts is still treated as subjective because it reflects stance, even when the stance is ambivalent (Bellaouar et al., 27 Aug 2025).

2. Source corpora and construction heuristics

AraDhati+ pools four public Arabic corpora spanning multiple domains and language varieties. ASTD contributes tweets in MSA and dialectal Arabic. LABR contributes book reviews with 1–5 ratings. HARD contributes hotel reviews in MSA and dialect. SANAD contributes news articles from seven categories, of which only Medical, Sports, and Technology are used in AraDhati+ as objective exemplars (Bellaouar et al., 27 Aug 2025).

Source corpus	Original annotation	AraDhati+ role
ASTD	Objective vs Subjective (Positive/Negative/Mixed)	Mixed source of subjective and objective
LABR	Ratings mapped to positive, negative, neutral	All included instances mapped to subjective
HARD	Ratings mapped to negative and positive; rating 3 removed in balanced version	All included instances mapped to subjective
SANAD	Single-labeled news categories	Medical, Sports, Technology mapped to objective

The original ASTD distribution is 3,315 subjective and 6,691 objective, with approximately 10,006 tweets, a maximum of 45 tokens per tweet, an average of 16, 160,206 total tokens, and a vocabulary of 38,743 (Bellaouar et al., 27 Aug 2025). LABR contains more than 63,000 reviews. HARD uses a balanced subset of 94,052 reviews. SANAD contains 194,797 articles across seven categories, from which AraDhati+ selects 32,500 objective articles (Bellaouar et al., 27 Aug 2025).

The construction strategy combines direct reuse and heuristic abstraction. Subjective examples consist of ASTD subjective labels together with all LABR and HARD reviews labeled positive, negative, or neutral. Objective examples consist of the selected SANAD texts. No manual re-annotation is reported; the labels are mapped automatically through clear, domain-informed heuristics (Bellaouar et al., 27 Aug 2025). This suggests that AraDhati+ is best understood as a high-coverage, heuristically labeled resource rather than a manually adjudicated gold-standard benchmark.

3. Balancing strategy, composition, and coverage

To counter ASTD’s imbalance and enlarge domain coverage, the dataset construction uses both oversampling and augmentation. The authors oversample ASTD subjective instances to construct balanced ASTD-only training for some model variants. They also augment the corpus with 32,500 SANAD objective articles and 32,500 subjective reviews equally drawn from LABR and HARD, 16,250 each, before including ASTD to form AraDhati+ (Bellaouar et al., 27 Aug 2025).

The final AraDhati+ dataset contains 77,916 instances, split into 62,332 training instances and 15,584 test instances under an 80/20 partition (Bellaouar et al., 27 Aug 2025). The per-source train/test breakdown is 10,332/2,584 for ASTD, 13,000/3,250 for LABR, 13,000/3,250 for HARD, and 26,000/6,500 for SANAD. The class distribution is constructed to be near-balanced overall via the 32,500 subjective versus 32,500 objective addition, while ASTD contributes mixed labels and makes the overall balance slightly objective-leaning relative to the exact 32.5k/32.5k core (Bellaouar et al., 27 Aug 2025).

AraDhati+ covers both MSA and dialectal Arabic, especially through HARD and ASTD, and spans tweets, book reviews, hotel reviews, and news. The dataset may contain occasional English words in tweets, although the extent of code-switching is not quantified. The paper notes that multilingual pretraining of XLM-R, without language embeddings, is beneficial for code-switching (Bellaouar et al., 27 Aug 2025). A plausible implication is that the dataset’s cross-domain construction was intended not only to enlarge sample size but also to expose models to heterogeneous discourse structures and register variation.

4. Preprocessing, representation, and released schema

The preprocessing pipeline applies cleaning and normalization before tokenization. The reported cleaning removes URLs, non-Arabic characters, punctuation marks, special characters, single-letter tokens, and other non-useful text artifacts (Bellaouar et al., 27 Aug 2025). Lexical normalization is applied where applicable, and stopword removal is explicitly avoided in order to preserve cues important for subjectivity and sentiment. Model-specific tokenizers are then used with truncation and padding to a maximum sequence length of 256.

The tokenization stack depends on the model family: XLMRobertaTokenizerFast for XLM-RoBERTa, BertTokenizerFast for AraBERT, and AraNizer for ArabianGPT (Bellaouar et al., 27 Aug 2025). Heuristics for social-media-specific tokens such as mentions, emojis, and hashtags are implicitly covered by cleaning and normalization, while links are stripped. Diacritics handling and specific Alef, Yaa, and Taa Marbuta normalization are not detailed.

The released CSV is structured around the fields Text, Class, Domain, Label, and Dataset. Text stores the raw Arabic string. Class stores the sentiment or tag, including POS, NEG, NEUTRAL for reviews and OBJ for objective news. Domain distinguishes Tweets, Book reviews, Hotel reviews, and Sports, Medical, or Technology news. Label stores the binary subjectivity target, and Dataset identifies the source corpus, ASTD, LABR, HARD, or SANAD (Bellaouar et al., 27 Aug 2025).

The repository is available at https://github.com/Attia14/AraDhati. The source datasets are free and publicly accessible under their own terms, but the explicit license for AraDhati+ itself is not specified in the paper, and users are directed to consult the repository and respect the source dataset licenses (Bellaouar et al., 27 Aug 2025).

5. Fine-tuned models and optimization regime

The paper fine-tunes three pre-trained LLMs in two scenarios: on oversampled ASTD only, denoted by _1 variants, and on the augmented AraDhati+ corpus, denoted by _2 variants (Bellaouar et al., 27 Aug 2025). XLM-RoBERTa is described as multilingual and pre-trained with masked language modeling on 2.5 TB of CommonCrawl. AraBERT is specified as an Arabic BERT with 12 layers, 768 hidden units, 12 attention heads, maximum length 512, and approximately 136M parameters. ArabianGPT-01B is a GPT-2 architecture adapted to Arabic with 12 decoder layers, approximately 134M parameters, and training on approximately 15.5GB of Arabic news (Bellaouar et al., 27 Aug 2025).

These yield six fine-tuned systems: AraSubjXLM-R_1, AraSubjXLM-R_2, AraSubjBERT_1, AraSubjBERT_2, AraSubjGPT_1, and AraSubjGPT_2. Training uses AdamW with mini-batch gradient descent, batch size 16, learning rates explored over $$\{5\mathrm{e}{-6}, 1.5\mathrm{e}{-5}, 2.0\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$, and epochs in $\{1, 2, 3, 5, 7\}$ (Bellaouar et al., 27 Aug 2025). The hardware is a GPU platform, with no further specification.

The loss is standard cross-entropy for two-class classification:

$$\mathcal{L} = -\sum_{n} \sum_{c \in \{0,1\}} y_{n,c} \log p_{n,c}$$

The formulation situates AraDhati+ squarely within a conventional Transformer fine-tuning regime rather than a task-specific architectural redesign. This suggests that the primary research contribution lies in dataset construction and cross-domain subjectivity framing, with model diversity and ensembling used to probe robustness across domains (Bellaouar et al., 27 Aug 2025).

6. Evaluation protocol, ensemble design, and empirical results

Evaluation uses a hold-out protocol with a stratified 20% test split per source. The paper also includes cross-domain generalization checks by evaluating models on subjective-only partitions, formed from LABR and HARD, and objective-only partitions, formed from SANAD (Bellaouar et al., 27 Aug 2025). Reported metrics are accuracy, precision, recall, and F1-score:

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$

$\text{Precision} = \frac{TP}{TP + FP}$

$\text{Recall} = \frac{TP}{TP + FN}$

$$F1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$$

Per-class breakdowns, confusion matrices, and confidence intervals are not provided (Bellaouar et al., 27 Aug 2025).

Two ensembles are defined by voting over the three fine-tuned base models. Decision_1 combines the _1 variants trained on oversampled ASTD. Decision_2 combines the _2 variants trained on AraDhati+ (Bellaouar et al., 27 Aug 2025). The implementation is described as majority voting over predicted labels. The paper also gives a probability-aggregation formulation,

$$s_c = \sum_i w_i \, p_i(c), \qquad \hat{y} = \arg\max_c s_c$$

with uniform weights $w_i = 1$, although majority voting is the reported method and no learned weights are given.

The headline result is that Decision_2 achieves 97.79% accuracy on the augmented AraDhati+ test set (Bellaouar et al., 27 Aug 2025). Model-wise, under oversampled ASTD training, AraSubjGPT_1 achieves the best accuracy on ASTD’s own test split at 87.78%; AraSubjXLM-R_1 obtains the best generalization to objective-only SANAD at approximately 98% accuracy; and AraSubjBERT_1 is strongest on subjective reviews, LABR plus HARD, at approximately 82% accuracy. Decision_1 reaches 95.62% on the ASTD test, 88.60% on the augmented test, and competitive performance on subjective-only and objective-only partitions at approximately 80.03% and approximately 94.37%, respectively (Bellaouar et al., 27 Aug 2025).

Under augmented AraDhati+ training, AraSubjGPT_2 and AraSubjBERT_2 obtain top accuracies on the ASTD test at approximately 86%, while all three _2 models exceed 99% accuracy on objective SANAD and subjective LABR plus HARD partitions (Bellaouar et al., 27 Aug 2025). Decision_2 attains 87.27% on the ASTD test and 97.79% on the augmented AraDhati+ test. The paper attributes the slight reduction on the original ASTD test, between $-1.28\%$ and $\{1, 2, 3, 5, 7\}$0 across models, to domain shift: the augmented training distribution includes reviews and news, which introduce cues less aligned with tweet subjectivity (Bellaouar et al., 27 Aug 2025).

7. Error profile, limitations, and research significance

The error analysis for Decision_2 identifies systematic failure modes. Of all errors, 30.40% are cases in which all three component models misclassify the same instance, indicating shared limitations rather than isolated model-specific failures (Bellaouar et al., 27 Aug 2025). Within that subset, three categories are reported: Mixed Tweets at 41%, Model errors at 33%, and Short Tweets at 26%.

Mixed Tweets combine factual clauses with opinionated segments, making binary sentence-level subjectivity difficult when factual scaffolding and evaluative content co-occur. Model errors include persuasive or news-like subjective posts incorrectly flagged as objective, as well as emotive short statements that are misread. Short Tweets provide too little context for reliable classification (Bellaouar et al., 27 Aug 2025). The paper further highlights Arabic-specific challenges, including rich morphology, affixes, clitics, dialect variation, orthographic variability, sarcasm, irony, and figurative language, especially in microtext with code-switching and hashtags.

The dataset limitations are stated explicitly: automatic label mapping, the treatment of neutral reviews as subjective, no human verification for the SANAD objectivity slice, potential residual imbalance due to ASTD inclusion, domain skew across news, reviews, and tweets, and the absence of inter-annotator agreement (Bellaouar et al., 27 Aug 2025). Quality-control limitations are also noted: deduplication and extra filtering beyond the stated cleaning are not documented. On the modeling side, the paper identifies sensitivity to domain shift, brittle behavior on short and noisy social content, and limited handling of mixed-content sentences.

Future directions include expanding sources to forums, encyclopedic content, and multi-genre corpora; adding manual annotations and adjudication for a gold-standard subjectivity dataset; introducing clause-level and aspect-level subjectivity; pursuing cross-lingual transfer and domain adaptation; adopting error-aware training through curriculum learning and contrastive objectives; and improving handling of irony and sarcasm (Bellaouar et al., 27 Aug 2025). Within Arabic NLP, AraDhati+ is therefore significant less as a closed benchmark than as a multi-domain resource that exposes the distinction between sentiment and subjectivity, and that makes domain-informed heuristic relabeling a concrete basis for large-scale Arabic subjectivity evaluation.