ArabEmoNet: Efficient Arabic Emotion Recognition

• ArabEmoNet is a hybrid architecture for Arabic speech emotion recognition that combines 2D CNN, BiLSTM, and temporal attention to extract fine-grained emotional cues.
• It leverages log-Mel spectrograms and a three-stage processing pipeline to achieve high accuracy (up to 99.46%) while maintaining a compact, efficient model size.
• The model addresses challenges of limited Arabic data with innovative augmentation strategies and efficient design, enabling real-time deployment in diverse applications.

ArabEmoNet is a lightweight, hybrid architecture specifically designed for robust Arabic speech emotion recognition, addressing challenges posed by limited data resources and underexplored representation learning in Arabic SER. The model achieves state-of-the-art accuracy and computational efficiency by combining two-dimensional convolutional neural networks (2D CNNs) with bidirectional LSTM (BiLSTM) and an integrated attention mechanism, operating directly on log-Mel spectrogram inputs rather than discrete MFCCs to preserve critical spectro-temporal emotional cues (Abouzeid et al., 1 Sep 2025).

1. Hybrid Model Architecture

ArabEmoNet employs a three-stage pipeline:

1. 2D Convolutional Layers:
   • Input signals are represented as log-Mel spectrograms.
   • Multiple Conv2D layers extract local time-frequency patterns, mathematically described by:

   $$F_\ell = \sigma(\text{Conv2D}(F_{\ell-1}, W_\ell, \text{padding}=p_\ell) + b_\ell)$$

   where $F_{\ell-1}$ is the previous feature map, $W_\ell$ the convolution kernels, $p_\ell$ the padding, $b_\ell$ the bias, and $\sigma$ the ReLU activation. - The use of 2D convolutions aligns with the 2D structure of Mel spectrograms, enabling the capture of nuanced patterns lost in 1D convolutional approaches.

2. Bidirectional LSTM Layer:

   • The sequentially ordered feature map from the CNN is processed by a BiLSTM.
   • The forward and backward passes produce hidden states for each time frame, concatenated as $h_t = [\overrightarrow{h_t} ; \overleftarrow{h_t}]$.
   • BiLSTM enhances modeling of temporal transitions and dependencies in emotional speech.
3. Temporal Attention Mechanism:
   • An attention layer computes context vector $c$ by weighting each BiLSTM output $h_t$:

   $e_t = \tanh(w_e^\top h_t + b_e)$

   $\alpha_t = \frac{\exp(e_t)}{\sum_k \exp(e_k)}$

   $c = \sum_t \alpha_t h_t$

- $w_e$ and $b_e$ are learnable parameters. - The attention mechanism emphasizes emotionally salient segments before the final classification layer.

The final context vector is processed by a fully connected layer to generate the logit outputs for emotion category prediction.

2. Log-Mel Spectrogram Feature Extraction

ArabEmoNet utilizes log-Mel spectrograms for input representation:

• Extraction Process:

  • Raw audio is converted using FFT (window size 2048, hop 256), yielding 128 Mel bands across 80–7600 Hz.
  • A Hann window function reduces spectral leakage per frame.
  • Spectrograms are log-scaled in decibels, referenced to maximum power.
• Advantages:
  • Mel spectrograms encode fine-grained temporal and frequency variations, essential for capturing emotional cues.
  • The 2D representation is optimally handled by 2D CNNs, unlike MFCCs, which are more limited and potentially discard subtle affective information.

This approach allows ArabEmoNet to model spectro-temporal dynamics critical for emotion recognition and avoids degradation associated with MFCC discretization and 1D convolutions.

3. Efficiency and Benchmark Performance

ArabEmoNet advances computational efficiency and accuracy for Arabic SER:

• Model Size:
  • Total parameters: ~1 million.
  • Comparison: HuBERT-base (95M), Whisper-small (244M).
  • ArabEmoNet is 90× smaller than HuBERT-base and 244× smaller than Whisper-small.
• Accuracy:
  • KSUEmotions dataset: 91.48%
  • KEDAS dataset: 99.46%
  • Outperforms competitors by several percentage points despite dramatically lower parameter counts.

Model	Parameters	Accuracy (KSUEmotion)	Accuracy (KEDAS)
ArabEmoNet	1M	91.48%	99.46%
HuBERT-base	95M	~87.04%	--
Whisper-small	244M	~85.98%	--

ArabEmoNet reconciles high accuracy with low computational footprint, making it suitable for deployment on resource-constrained platforms.

4. Technical Innovations and Methodological Advancements

Key technical improvements introduced by ArabEmoNet:

• Direct 2D convolution on Mel spectrograms as opposed to MFCC features and 1D convolution, allowing better exploitation of spectro-temporal structure.
• Temporal attention atop BiLSTM for selective aggregation of emotional frames, improving discriminability.
• Parameter efficiency supporting real-time, embedded, and mobile-friendly Arabic SER systems.

Data augmentation methods, including SpecAugment and additive white Gaussian noise, further enhance robustness and generalization.

5. Application Spectrum and Practical Impact

ArabEmoNet is engineered for real-world Arabic speech emotion recognition in various domains:

• Human-computer interaction (affective conversational agents, smart assistants, call centers).
• Mobile and embedded systems, where model size and inference speed are constraining factors.
• Analytics for customer sentiment and educational technology requiring real-time emotional feedback in Arabic.

Its superior accuracy and accessibility for Arabic positions ArabEmoNet as a backbone for future multilingual and multimodal emotion-aware AI systems.

6. Limitations and Addressed Challenges

ArabEmoNet’s development contended with several technical limitations:

• Variable-length input handling: Uses zero-padding to unify batch sizes, which may introduce non-informative frames—identified as a current limitation.
• Data scarcity in Arabic: Mitigated by data augmentation, with a suggested future direction in multi-lingual scaling.
• Generalization: Augmentation and compact modeling confer robustness, though broader cross-cultural dataset validation is warranted.

7. Future Directions

Prospective enhancements envisioned for ArabEmoNet:

• Training on larger and multilingual datasets for broadened generalizability.
• Refinements to input padding and variable-length sequence modeling.
• Additional regularization and augmentation strategies for further robustness.
• Integration of text and visual modalities for comprehensive affective computing systems.

These directions reflect the model’s adaptability for both continued Arabic SER research and cross-lingual emotional speech processing architectures.

---

ArabEmoNet represents an optimized, hybrid approach in Arabic speech emotion recognition, combining convolutional, sequential, and attention-based modules atop log-Mel spectrogram features to robustly and efficiently extract emotional states. Its architecture, empirical performance, and technical judiciousness offer a scalable foundation for both current and future research in Arabic affective computing and practical deployments (Abouzeid et al., 1 Sep 2025).