ArSarcasm-v2: Arabic Sarcasm Detection Dataset

• ArSarcasm-v2 is an Arabic Twitter dataset with labels for sarcasm, sentiment, and dialect, enabling multi-task analysis.
• The dataset comprises 12,548 training and 3,000 testing tweets, originally used in the EACL WANLP-2021 Shared Task.
• Advanced preprocessing and transformer-based models reveal task dependencies, enhancing sarcasm and sentiment detection.

Searching arXiv for the specified papers to ground the article in the cited literature. ArSarcasm-v2 is an Arabic Twitter dataset used for sarcasm detection and sentiment analysis in EACL WANLP-2021 Shared Task 2, and later for joint modeling of sentiment, sarcasm, and dialect. In the shared-task setting, it comprises 12,548 Arabic tweets for training/validation and a held-out 3,000 tweets for testing. In later work, the same split is treated as a 15,548-tweet resource with labels for sentiment, sarcasm, and dialect, which makes it useful both as a task-specific evaluation set and as a substrate for multi-task learning over correlated social-media phenomena (Wadhawan, 2021, Kaseb et al., 2023).

1. Shared-task role and corpus definition

ArSarcasm-v2 is the official training set for EACL WANLP-2021 Shared Task 2, subtask 1 for sarcasm detection and subtask 2 for sentiment analysis. The shared-task formulation distinguishes a binary sarcasm label and a three-way sentiment label. For sarcasm detection, the labels are TRUE for a sarcastic tweet and FALSE for a non-sarcastic tweet. For sentiment, the labels are NEG, NEU, and POS.

Later work uses the same data split as a 15,548-tweet collection consisting of 12,548 training examples and 3,000 test examples, and treats each tweet as carrying three labels: sentiment, sarcasm, and dialect. In that formulation, dialect takes one of five values: MSA, EGY, Gulf, LEV, or NOR. This makes ArSarcasm-v2 not merely a benchmark for isolated sarcasm detection, but also a dataset for studying dependencies among sentiment, sarcasm, and regional language variation (Wadhawan, 2021, Kaseb et al., 2023).

Partition	Tweets	Use
Training set	12,548	Training/validation in the shared-task split
Test set	3,000	Held-out shared-task test set
Total	15,548	Corpus size as used in later work

2. Annotation scheme and label distributions

The annotation guidelines reported for the shared task specify that each tweet was examined by multiple annotators. A tweet is marked TRUE if at least two annotators agree that it conveys sarcasm; otherwise it is FALSE. Annotators were instructed to look for explicit markers of irony or incongruity between literal content and real-world facts or expectations. Sentiment annotations followed standard polarity guidelines: NEG for negative affect, POS for positive affect, and NEU when no clear polarity.

On the 12,548-tweet training set, the label counts reported in later work are 2,180 positive, 5,747 neutral, and 4,621 negative tweets for sentiment; 2,168 sarcastic and 10,380 non-sarcastic tweets for sarcasm; and 8,562 MSA, 2,675 EGY, 644 Gulf, 624 LEV, and 43 NOR tweets for dialect. The same study reports cross-task regularities: approximately 90% of sarcastic tweets are labeled NEG, approximately 50% of MSA tweets are labeled NEU, and approximately 50% of EGY tweets are labeled NEG. These correlations motivate architectures in which sentiment prediction is explicitly informed by sarcasm and dialect predictions (Wadhawan, 2021, Kaseb et al., 2023).

Task	Label	Count
Sentiment	POS	2,180
Sentiment	NEU	5,747
Sentiment	NEG	4,621
Sarcasm	Sarcastic	2,168
Sarcasm	Non-sarcastic	10,380
Dialect	MSA	8,562
Dialect	EGY	2,675
Dialect	Gulf	644
Dialect	LEV	624
Dialect	NOR	43

For model development in the 2021 shared-task system, the 12,548 tweets were re-split into 90% train and 10% dev, yielding 11,293 training tweets and 1,255 development tweets. In that re-split, sarcasm counts were 9,356 FALSE and 1,937 TRUE in train, and 1,024 FALSE and 231 TRUE in dev. Sentiment counts were 4,139 NEG, 5,197 NEU, and 1,957 POS in train, and 482 NEG, 550 NEU, and 223 POS in dev (Wadhawan, 2021).

3. Preprocessing and morphological segmentation

One reported treatment of ArSarcasm-v2 applies four successive transformations to each tweet before transformer finetuning. The first stage is cleaning and noise removal, which strips HTML line-break tags such as <br> and other markup, deletes non-standard emoticons or non-text unicode symbols, collapses character repeats beyond two, and normalizes extra whitespace. The stated purpose is to remove social-media artifacts that carry little or no signal for sarcasm or sentiment and to reduce token-vocabulary sparsity.

The second stage is Farasa segmentation for model variants that expect segmented input. Farasa segmentation splits clitics and affixes from stems; the summary gives the example $$\text{Al+kitābuhu} \rightarrow [\text{Al+}][\text{kitāb}][\text{+hū}]$$. The segmentation scheme is based on the open-source Farasa segmenter and applies affix lookup, statistical stem isolation, and recoding of enclitics. It is explicitly described as morphological tokenization rather than character-level heuristics. The paper reports use of Farasa’s default settings and does not report custom thresholds or beam sizes. It also notes that Farasa internally relies on a Viterbi-style decoding to choose the highest-scoring segmentation path, although those equations are not restated.

The third stage is whitespace insertion around mixed scripts, with spaces inserted before and after English letters or digits adjacent to Arabic script, and around parentheses. The fourth stage is placeholder replacement, mapping URLs to [<رابط>], emails to [~<بريد>], and user mentions to [<مستخدم>]. These operations abstract away arbitrary strings while retaining their functional presence. A plausible implication is that the preprocessing is designed not only for normalization, but also for preserving tweet-level discourse cues that may remain relevant after lexical identity is removed (Wadhawan, 2021).

4. Transformer-based baselines and segmentation effects

The 2021 shared-task system feeds each cleaned, and optionally segmented, tweet into a Hugging-Face Arabic BERT or ELECTRA tokenizer. The input sequence is wrapped as [CLS] … tokens … [SEP] and truncated or padded to max_length = 256 tokens. The model’s original subword vocabulary and positional embeddings are used without modification, and no additional task-specific embeddings or adapters are injected. The experiments compare multiple variants of two transformer-based models, AraELECTRA and AraBERT.

For sarcasm detection, the reported validation results permit a direct comparison between pre-segmented and non-segmented runs. On subtask 1, AraBERTv0.2-base without segmentation reaches a weighted $F_1$ of 84.62%; AraBERTv1-base with segmentation reaches 84.86%; and AraBERTv2-base with segmentation reaches 85.55%. The summary states that moving from unsegmented AraBERTv0.2-base to segmented AraBERTv1-base yields approximately $+0.24$ points in $F_1$, while AraBERTv2-base reaches approximately $+0.9$ points over the unsegmented variant. It further states that, while the absolute effect of Farasa segmentation is under 1 $F_1$ point, it consistently improves model robustness, especially on the minority sarcastic class.

The final shared-task submission was ranked seventh in sarcasm detection and fourth in sentiment detection. On the official test set, the system obtained 58.72% for subtask 1, where the official metric is $F_1$ on the TRUE class, and 65.31% for subtask 2, where the official metric is Macro-$F_1$ on POS/NEG (Wadhawan, 2021).

5. Multi-task exploitation of sarcasm, sentiment, and dialect

A later line of work treats ArSarcasm-v2 as a jointly labeled dataset and introduces SAIDS, a system that predicts sentiment, sarcasm, and dialect of Arabic tweets. SAIDS uses MARBERTv2 as a LLM to generate a sentence embedding, then passes that embedding to sarcasm and dialect models, concatenates their outputs with the original embedding, and feeds the result to a sentiment classifier. Tweets are tokenized by the MARBERTv2 tokenizer and truncated or padded to a maximum length of 128.

The architecture is specified as follows. A tokenized tweet is encoded by MARBERTv2 into a pooled sentence embedding of size $D = 768$. Two first-layer classifiers then operate on that embedding: a sarcasm classifier mapping $\mathbb{R}^{768} \rightarrow \mathbb{R}^{2}$ and a dialect classifier mapping $F_1$0. Both are shallow multi-layer nets with zero hidden layers, namely a single linear layer followed by Softmax. The final sentiment classifier takes the concatenation of the 768-dimensional MARBERT embedding, the 2-dimensional Softmax output of the sarcasm model, and the 5-dimensional Softmax output of the dialect model, giving $F_1$1 input units, followed by one linear layer and a three-way Softmax.

The losses are binary cross-entropy for sarcasm, categorical cross-entropy for dialect, and categorical cross-entropy for sentiment:

$F_1$2

$F_1$3

$F_1$4

The total training objective is the sum of the three losses,

$F_1$5

with $F_1$6 in practice. Training uses Adam, a learning rate tuned in $F_1$7, typically 5 epochs, up to 20 epochs for the smallest learning rate, batch size 32, and no additional data augmentation or dropout or weight decay. The best training sequence, denoted Seq 1, consists of one epoch training only the sarcasm and dialect heads with MARBERT frozen, followed by four epochs training all three heads and MARBERT together. The study also introduces limited backprop, in which errors from the sentiment loss are prevented from flowing into the final output layers of the sarcasm and dialect heads; the stated reason is to ensure that each head learns its own task robustly (Kaseb et al., 2023).

6. Evaluation protocols, results, and interpretive significance

The 2023 study defines the evaluation metrics explicitly. For a single class,

$F_1$8

For sentiment, the reported metric is FPN, the average of $F_1$9 over positive and negative classes:

$+0.24$0

For sarcasm, the metric is FSar, the $+0.24$1 score on the sarcastic class only. For dialect, the metric is WFS, the weighted average $+0.24$2 over all five dialects, with weights proportional to class support.

On the shared 3,000-tweet test set, SAIDS reports 75.98 FPN for sentiment analysis, 59.09 FSar for sarcasm detection, and 71.13% weighted $+0.24$3 for dialect identification. In the same comparison table, the reported values are 71.60/58.41 for Baseline 1, 72.53/58.61 for Baseline 2, 73.11/58.62 for Baseline 3, 74.80/60.00 for el-mahdaouy et al. (2021), 73.92/61.27 for DeepBlueAI (song et al. 2021), 73.21/56.62 for Abdel-Salam (2021), and 72.55/58.72 for Wadhawan (2021). The study states that SAIDS outperforms the state-of-the-art model for the sentiment analysis task, while the highest listed sarcasm score in that table is DeepBlueAI’s 61.27 FSar.

Ablation studies on a random 10% validation split of the training data further characterize the dataset’s task dependencies. For sentiment prediction, the uninformed baseline yields 72.40% FPN, informing with sarcasm only yields 73.67%, informing with dialect only yields 74.41%, and informing with both yields 75.23%, a gain of +2.83 percentage points over the uninformed baseline. Zero hidden layers in each head perform best at 75.23%, compared with 74.90% for one hidden layer and 74.89% for two. Softmax on the heads yields 75.23%, whereas raw scores yield 72.15%. For backpropagation control, full limit gives 74.23%, partial limit gives 74.89%, and unlimited end-to-end training gives 72.31%. For training sequence, all tasks together from epoch 1 gives 74.35%, Seq 1 gives 75.23%, Seq 2 gives 73.49%, and Seq 3 gives 73.01%. A proof-of-concept Random Forest experiment increases FPN from 59.36% to 62.34% when informed by sarcasm and dialect outputs. This suggests that ArSarcasm-v2 supports architectures that exploit label interdependence rather than treating sentiment as conditionally independent of sarcasm and dialect (Kaseb et al., 2023).

A common simplification is to describe ArSarcasm-v2 solely as a binary sarcasm corpus. The reported evidence does not support that reduction. In the shared-task setting it is simultaneously a sarcasm-detection and sentiment-analysis dataset, and in later work it is additionally used as a dialect-labeled resource. Another simplification is to treat preprocessing as incidental. The 2021 experiments report consistent, if modest, gains from Farasa-based segmentation, whereas the 2023 experiments show materially larger gains from explicitly modeling correlations among sarcasm, sentiment, and dialect. Taken together, these results position ArSarcasm-v2 as a dataset in which both Arabic-specific preprocessing and multi-task dependency structure are empirically consequential (Wadhawan, 2021, Kaseb et al., 2023).