EHSAN: Hybrid ABSA Pipeline in Healthcare

• EHSAN is a hybrid framework for Arabic aspect-based sentiment analysis that addresses dialect variation, limited annotations, and the need for explainability in healthcare feedback.
• It combines ChatGPT-generated pseudo-labelling, targeted human review, and transformer-based models like AraBERT and DistilBERT to enhance annotation quality and transparency.
• Evaluations indicate marginal performance loss without full human review and improved results using coarser aspect taxonomies, underscoring its practical utility in low-resource settings.

EHSAN is a hybrid data-centric pipeline for Arabic aspect-based sentiment analysis (ABSA) in healthcare settings. It addresses the challenges posed by dialect variation, limited annotated resources, and lack of explainability in automated sentiment analysis systems for Arabic-language patient feedback. The framework systematically integrates ChatGPT-generated pseudo-labelling, targeted human review, and transformer-based modeling, culminating in the first explainable Arabic ABSA dataset with per-sample rationales for healthcare themes (Alamoudi et al., 4 Aug 2025).

1. Hybrid Pseudo-Labelling and Review Pipeline

EHSAN employs a staged, “pseudo-label plus review” workflow to construct a robust annotated corpus:

• Data Collection and Pre-processing: Patient reviews (n=5,000) were scraped from three Saudi hospitals (Riyadh, Jeddah, Dammam). Texts were normalized via regex-based sentence segmentation and the AraBERT preprocessor to ensure homogeneity of Arabic script and remove noise (diacritics, emojis, elongations, extraneous whitespace).
• Topic-Constrained Sentence Segmentation: GPT-4-mini, via prompt engineering, split each review into semantically coherent, single-topic sentences. Coverage was validated using BERTopic (yielding ≈25 data-driven topics) and compared with Stanza’s rule-based segmentation; ChatGPT splits were adopted, resulting in 6,000 sentences.
• ChatGPT Pseudo-Labelling: Each sentence was processed with a few-shot prompt to assign (1) an aspect label from a 17-class taxonomy, (2) sentiment ∈ {Positive, Negative, Neutral}, and (3) a free-text rationale (initially in English, then translated to Arabic). Sentence-level IDs enabled traceability from raw data to annotation.
• Tiered Human Verification: Two native-Arabic annotators with specialized training on the taxonomy reviewed labels. Inter-annotator reliability yielded Cohen’s κ = 0.873 on a 5% subsample. Three dataset variants were produced by varying the proportion of human-reviewed labels:
  • Fully Supervised (FSD): 100% human-reviewed
  • Semi-Supervised (SSD): 50% human-reviewed
  • Unsupervised (USD): 0% human review (purely ChatGPT labels)
• Final Dataset Splitting and Label Schemes: Sentences were partitioned into train (3,583), dev (1,189), and test (1,190) subsets. Aspect labels were provided both in the original 17-category granularity and a merged 6-class taxonomy, facilitating analysis of granularity effects.

2. Annotation Schema and Rationales

Each sentence in EHSAN is annotated with two categorical fields:

• Aspect: Either one of 17 fine-grained healthcare-related classes (e.g., Medical_and_Nursing_Staff, Billing_and_Finance, Privacy) or one of 6 coarser-level groupings.
• Sentiment: One of {Positive, Negative, Neutral}.

A distinctive feature is the inclusion of model-generated rationales—free-text justifications for each (aspect, sentiment) assignment. These rationales, supplied by ChatGPT and translated into Arabic when needed, serve to enhance transparency and facilitate post-hoc auditing of annotation decisions.

3. Dataset Construction and Labeling Variants

The resulting dataset comprises 6,000 sentences, each mapped to an aspect and sentiment, with distinct subsets:

Subset	Size	Label Verification	Notes
Train	3,583	FSD, SSD, USD variants	Human review ratio varies
Dev	1,189	100% human-verified	Consistent validation set
Test	1,190	100% human-verified	Consistent evaluation set

This design enables controlled experiments on annotation quality and its downstream impact. The 17→6 class grouping also supports direct analysis of granularity effects on model robustness.

4. Transformer-Based Aspect and Sentiment Classification

Two transformer architectures were fine-tuned for both aspect classification and sentiment analysis:

• AraBERT v0.2 (Large): Pre-trained on Modern Standard and dialectal Arabic, employing a custom tokenizer optimized for Arabic morphology.
• DistilBERT (Multilingual): A 6-layer, distilled BERT supporting over 100 languages, using a multilingual WordPiece tokenizer.

Key training settings included batch size 4, learning rate $10^{-5}$, 5 epochs, input sequence length up to 128 tokens, AdamW optimizer, and early stopping on dev macro-F1. Experiments were run for (a) 17-class aspects, (b) 6-class aspects, and (c) 3-class sentiment tasks, using PyTorch and HuggingFace infrastructure.

5. Evaluation Protocols and Performance Analysis

Evaluation metrics prioritize class-imbalance robustness:

• Macro-Precision, Macro-Recall, Macro-F1: Computed by averaging class-wise scores equally across all classes $K$:

$$\text{Macro-Precision} = \frac{1}{K} \sum_{n=1}^K \text{Precision}_n$$

$$\text{Macro-Recall} = \frac{1}{K} \sum_{n=1}^K \text{Recall}_n$$

$$\text{Macro-F1} = \frac{2 \cdot \text{Macro-Precision} \cdot \text{Macro-Recall}}{\text{Macro-Precision} + \text{Macro-Recall}}$$

• Overall Accuracy is also reported.

17-Class Aspect Classification

• AraBERT achieved macro-F1 = 0.66 (FSD), with only a ΔF1 = 0.02 drop (to 0.64) for USD. DistilBERT trailed by 0.05–0.07 points.
• Cohen's κ for annotators was 0.873, indicating strong reliability.
• Statistical significance for FSD vs. USD for AraBERT: $p = 0.021$; all other differences not significant.

6-Class Aspect Classification

• AraBERT FSD macro-F1 increased to 0.78; human-review impact (FSD vs. USD) diminished to ΔF1 = 0.01.
• DistilBERT achieved macro-F1 = 0.69 (FSD).
• The reduction in aspect granularity yields F1 gains of +0.08 to +0.13, depending on the model.

Sentiment Classification

• AraBERT exhibited macro-F1 = 0.84 (both FSD and USD), with accuracy 0.92–0.93.
• DistilBERT consistently scored macro-F1 ≈ 0.67–0.67, trailing AraBERT by ≈0.17.

6. Patterns, Granularity Effects, and Interpretability

Reducing aspect taxonomy from 17 to 6 categories significantly improved all metrics (AraBERT: F1 from 0.66→0.78, USD 0.64→0.77; DistilBERT similarly benefited). This supports the proposition that coarser classes lower sample complexity and attenuate label noise.

A notable finding is the robustness of ChatGPT-generated labels: the absence of human review led to only marginal F1 loss (≤0.02) in the most challenging aspect setting, and virtually none for 6-class and sentiment tasks.

Model-generated rationales improved annotator trust and provide an operational basis for explainable ABSA, offering justifications that can be systematically audited.

7. Prospects and Limitations

Key directions cited for extending EHSAN include:

• Generalization across hospitals, Arabic dialects, and healthcare regions.
• Further prompt engineering to minimize residual LLM labelling errors and bias.
• Multi-task models enabling simultaneous label prediction and natural language rationale generation.
• Auditing rationales for fairness across dialects and patient groups.
• Dynamic taxonomy adaptation and provenance tracking for full traceability of annotation decisions.
• Integration of privacy-preserving similarity search for sensitive patient feedback.

This suggests that the hybrid pseudo-labelling paradigm provides an effective template for building explainable, data-driven ABSA resources in low-resource and domain-sensitive contexts (Alamoudi et al., 4 Aug 2025).