Active Learning for Arabic Sentiment Analysis

• The paper presents an active learning framework that integrates deep learning models with LLM-assisted annotation to reduce labeling effort and enhance sentiment classification in Arabic.
• It utilizes uncertainty sampling and sequential architectures like RNN, LSTM, and GRU to selectively annotate informative samples and manage linguistic complexities.
• Evaluation on diverse datasets confirms the framework’s effectiveness in handling Modern Standard Arabic, dialects, and domain-specific variations with improved annotation efficiency.

An active learning framework for Arabic sentiment analysis operationalizes the goal of maximizing sentiment classification accuracy with minimal human annotation effort in a linguistically complex, resource-scarce context. This field has evolved from lexicon-based and classical machine learning techniques to deep learning and, most recently, LLM-assisted approaches for annotation. Frameworks are evaluated primarily on their ability to address the challenges associated with Modern Standard Arabic (MSA), dialectal variations, and domain-specific data, while efficiently leveraging high-capacity classifiers and scalable annotation protocols.

1. Framework Architecture and Workflow

The prototypical active learning framework for Arabic sentiment analysis comprises the following phases (Refai et al., 27 Sep 2025):

1. Preprocessing: Raw dataset preparation involves normalization, tokenization, and standardization of Arabic text. Tasks include removal of diacritics, normalization of Hamzah variants, de-duplication, and handling of morphology.
2. Model Initialization: A small seed set of annotated samples is used to train an initial classifier. Deep sequential models—Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRU)—are favored for their capacity to learn both short- and long-range dependencies in Arabic text.
3. Uncertainty-Driven Querying: At each iteration, the trained model predicts on an unlabeled pool and selects the most uncertain samples for annotation. Uncertainty is typically computed via entropy-based metrics over the model’s output probabilities.
4. Annotation Loop: Selected samples are annotated either by humans or, in the LLM-in-the-loop paradigm, by a high-performing LLM. The LLM is chosen empirically for each dataset from a candidate pool (e.g., GPT-4o, Claude 3 Sonnet, DeepSeek Chat) based on closed-sample classification accuracy.
5. Model Update and Iteration: Newly labeled data are added to the training set, the model is retrained, and the cycle repeats until a target performance criterion or annotation budget is reached.

This process exploits the fact that not all data points contribute equally to the classifier’s learning; by focusing annotation effort on informative examples, the active learning loop drives rapid accuracy gains with few labels.

2. Deep Learning Architectures for Sequential and Morphologically Rich Data

The framework employs deep learning models specifically adapted to process Arabic’s sequential and morphological properties (Refai et al., 27 Sep 2025):

• RNN: Suitable for sequence modeling, but less effective for long dependencies due to vanishing gradients.
• LSTM: Incorporates gates (input, output, forget) to retain information over extended context windows; mitigates vanishing gradients and yields superior results on long, complex sentences.
• GRU: A streamlined alternative to LSTM, with fewer parameters but similar efficacy for sequence representation.

For all architectures, features are extracted from cleaned and tokenized Arabic text, and models are evaluated via standard metrics (accuracy, F1, precision, recall). LSTM models, in particular, have achieved the highest accuracy across varied datasets and annotation schemes.

Architecture	Best-Observed Accuracy (Examples)	Suitability for Arabic Data
RNN	Moderate (e.g., 77-85%)	Captures sequential, short-term patterns
GRU	High (e.g., 79-87%)	Efficient with fewer parameters
LSTM	Highest (up to 93% on some sets)	Excels on long sentences, morphology

LSTM demonstrates robustness for both MSA and dialectal Arabic, notably in larger and more heterogeneous datasets.

3. Labeling Strategies: Human and LLM-in-the-Loop Annotation

The active learning loop relies on strategic sample annotation:

• Human Labeling: Traditional, high-quality but labor-intensive; baseline against which other methods are measured.
• LLM-Assisted Annotation: Five high-performing multilingual LLMs are empirically benchmarked for each dataset (Refai et al., 27 Sep 2025). The top performer for a given dataset (e.g., GPT-4o for Hunger Station reviews, DeepSeek Chat for MASAC) is used to generate sentiment labels for uncertainty-selected samples.
  • The LLM receives the raw Arabic text and a prompt specifying the sentiment classification task.
  • The best LLM is identified based on its accuracy in a held-out, closed sample from the target dataset prior to deployment in the loop.

The LLM-in-the-loop modality enables rapid annotation at scale, reduces cost, and—in certain domains—achieves competitive or superior accuracy relative to manual annotation.

Dataset	Best LLM	LLM Accuracy	# Labels (LLM/Human) to Baseline
Hunger Station	GPT-4o	93%	450 / 2,700
AJGT (Jordanian)	Claude 3 Sonnet	86%	400 / ~900
MASAC (multi-domain)	DeepSeek Chat	82%	650 / 1,080

The number of labeled samples needed to reach or surpass baseline accuracy is substantially lower for LLM-assisted labeling, showcasing the efficiency advantages of this approach.

4. Dataset Coverage and Linguistic Diversity

The framework’s empirical validation covers diverse domains and Arabic varieties (Refai et al., 27 Sep 2025):

• Hunger Station (reviews, MSA + local dialects): Over 51,000 reviews, labeled subset ~5,000. Reflects code-switching and service review language.
• AJGT (Arabic Jordanian General Tweets): 1,800 tweets with balanced class distribution, blending Modern Standard Arabic and Jordanian dialect. Represents informal, social media communication.
• MASAC: 2,000+ sentences from domains including education, technology, and health. Primarily MSA with dialect admixture.

Testing across these datasets validates the generalization and robustness of the framework, including its ability to handle both formal and informal (dialectal) Arabic content.

5. Performance Evaluation and Efficiency

Performance is measured using standard classification metrics. The key findings (Refai et al., 27 Sep 2025):

• LSTM models with LLM-labeling reach or surpass human-labeling accuracy with a significantly reduced labeled set (Hunger Station: 93% accuracy with 450 samples using GPT-4o, compared to 2,700 samples with human annotation).
• On MASAC, DeepSeek Chat yields 82% accuracy with 650 samples, matching human annotation with 1,080 samples.
• The annotation cost reduction is realized across all datasets, with LLM-in-the-loop methods converging to baseline accuracy in fewer active learning iterations.
• Accuracy is computed as:

$$\text{Accuracy} = \frac{\mathrm{TP} + \mathrm{TN}}{\mathrm{TP} + \mathrm{TN} + \mathrm{FP} + \mathrm{FN}}$$

where TP, TN, FP, FN denote true positives, true negatives, false positives, and false negatives, respectively.

Statistical analysis confirms that both annotation strategies improve model performance iteratively, but LLM-assisted approaches can often accelerate convergence.

6. Model Selection, Adaptation, and Future Scope

The approach includes an empirical selection phase to match each dataset with the highest-performing LLM, optimizing for both text type (e.g., reviews vs. tweets) and dialectal/language-specific nuances (Refai et al., 27 Sep 2025). This tuning ensures that model and annotation choices are tailored to corpus characteristics.

Looking forward, the paper identifies several extensions:

• Expansion to additional dialects and domains to further test framework generalizability.
• Fine-tuning LLMs with in-domain Arabic data to improve sensitivity to local linguistic phenomena.
• Experimentation with alternative uncertainty-based querying strategies (e.g., committee-based disagreement sampling) to maximize informativeness of selected instances.
• Integration of pre-trained transformers (AraBERT, MARBERT) and alignment with feature-based or explainable AI models for added interpretability and efficiency.

7. Significance and Impact

This active learning framework, through LLM-in-the-loop integration, achieves substantial gains in annotation efficiency and makes state-of-the-art Arabic sentiment analysis accessible even with limited labeled data. By uniting high-performing sequential models, uncertainty sampling, and dynamic labeler selection, it systematically addresses the bottlenecks of data scarcity and annotation cost in Arabic NLP. The comprehensive benchmarking across formal and dialectal datasets demonstrates its applicability for both academic research and industrial-scale sentiment monitoring in diverse Arabic contexts (Refai et al., 27 Sep 2025).