Arabic TTS with FastPitch: Reproducible Baselines, Adversarial Training, and Oversmoothing Analysis

Abstract: Arabic text-to-speech (TTS) remains challenging due to limited resources and complex phonological patterns. We present reproducible baselines for Arabic TTS built on the FastPitch architecture and introduce cepstral-domain metrics for analyzing oversmoothing in mel-spectrogram prediction. While traditional Lp reconstruction losses yield smooth but over-averaged outputs, the proposed metrics reveal their temporal and spectral effects throughout training. To address this, we incorporate a lightweight adversarial spectrogram loss, which trains stably and substantially reduces oversmoothing. We further explore multi-speaker Arabic TTS by augmenting FastPitch with synthetic voices generated using XTTSv2, resulting in improved prosodic diversity without loss of stability. The code, pretrained models, and training recipes are publicly available at: https://github.com/nipponjo/tts-arabic-pytorch.

• The paper introduces novel cepstral-domain metrics and adversarial training to significantly reduce oversmoothing in FastPitch-based Arabic TTS models.
• The methodology employs a modular two-stage TTS pipeline and integrates synthetic multi-speaker data using XTTSv2 for enhanced prosody control.
• The findings highlight that objective cepstral metrics and reproducible baselines can standardize TTS evaluation for low-resource languages.

Arabic TTS with FastPitch: Advancements in Baselines, Adversarial Training, and Oversmoothing Analysis

Introduction

The paper "Arabic TTS with FastPitch: Reproducible Baselines, Adversarial Training, and Oversmoothing Analysis" (2512.00937) presents a technically rigorous investigation into neural text-to-speech (TTS) for Arabic, addressing the data scarcity and complexity inherent to the language, particularly in phonology and dialectal variation. The work introduces reproducible FastPitch-based Arabic TTS baselines, proposes cepstral-domain oversmoothing metrics, and evaluates an adversarial spectrogram loss mechanism to mitigate oversmoothing. Multi-speaker synthesis is also explored via integration of synthetic voices generated through XTTSv2, providing controlled prosodic diversity.

Corpus Design and Feature Statistics

The paper leverages the Arabic Speech Corpus (ASC), which comprises 1,813 utterances spoken by a single male voice, and synthesizes three additional voices with XTTSv2 for multi-speaker training. Data analysis reveals systematic statistical mismatches between train and test splits—the training set incorporates many pseudo-word phoneme-rich utterances, while the test set contains exclusively natural sentences. This difference manifests in utterance duration, number of phonemes, speaking rate, and pitch distribution.

Figure 1: Distribution contrast of utterance-level features between ASC train and test splits, exposing domain mismatch relevant to metric interpretation.

TTS Pipeline and Model Architecture

The TTS pipeline follows the canonical two-stage approach, converting Arabic text to phonemes, encoding phoneme IDs, and generating log-mel spectrograms via FastPitch. A HiFi-GAN neural vocoder synthesizes waveforms from predicted spectrograms. Such modular design promotes efficient training, stability, and flexible model experimentation. FastPitch is configured with explicit duration, pitch, and energy predictors, enabling finer prosodic control.

Oversmoothing in Mel-Spectrogram Prediction

Despite effective convergence, standard regression (L1/L2) losses bias models toward mean spectral predictions, inherently dampening fine spectral detail—a phenomenon termed oversmoothing. The paper provides a spectral-domain analysis showing L2’s tendency to attenuate unpredictable details, particularly in bins with high phase variance. Cepstral metrics—High-Quefrency Energy Ratio (HQER), Cepstral Slope (CSlope), Cepstral Centroid (CCentroid), and Cepstral Rolloff (CRoll95)—are introduced to directly quantify oversmoothing in the mel-cepstrogram domain.

Figure 2: Log-mel spectrogram examples comparing reference, baseline, and adversarial system outputs, highlighting the preservation of fine structure with adversarial training.

Figure 3: HQER metric visualization; shaded region marks energy beyond the high-quefrency cutoff, reflecting spectral richness.

Quantitative Analysis of Regression and Adversarial Training

Model evaluation demonstrates rapid convergence under standard L2 training, with declining L1/L2/SConv errors and improved pitch/rate metrics. However, pitch and duration predictors both exhibit reduced dynamic range, compressing variance relative to ground truth. The introduction of a lightweight LS-GAN-based adversarial spectrogram loss stabilizes training (via spectral normalization and feature matching), and consistently reduces all cepstral oversmoothing metrics, leading to spectrograms with sharper contrast and increased high-frequency content.

Figure 4: Histogram comparison between alignment network and predictor frame durations; predictor compresses the long-duration tail, indicative of regression bias.

Figure 5: Predicted pitch distribution displays lower mean and variance than ground-truth, aligning with observed regression-induced compression.

Figure 6: Standard loss training curve for FastPitch.

Figure 7: Comparison of FastPitch loss curves with and without adversarial training; adversarial objectives remain stable.

Figure 8: Mel spectrogram distance metrics, illustrating marginal increases under adversarial loss due to improved spectral detail representation.

Multi-Speaker Arabic TTS with Synthetic Data

By leveraging XTTSv2 to generate synthetic speakers (male/female), the paper extends FastPitch to multi-speaker settings. Metrics show differences aligned with speaker physiology (e.g., female embeddings yield higher HQER and CCentroid due to elevated $f_0$), but overall performance for the original ASC speaker remains uncompromised, confirming no negative transfer. All cepstral metrics converge robustly across speakers, with adversarially trained models consistently outperforming regression-only baselines in oversmoothing reduction.

Figure 9: Speaker-wise metric curves for multi-speaker FastPitch, demonstrating uniform convergence and stability.

Figure 10: Mel-spectrogram distance comparisons reaffirm preservation of baseline performance in multi-speaker configuration.

Implications and Future Directions

The integration of cepstral-domain metrics provides an interpretable, objective framework for oversmoothing analysis—a significant advancement for TTS model diagnostics. The success of adversarial spectrogram loss validates generative adversarial regularization as both computationally economical and effective for realistic spectrum synthesis in Arabic TTS. Moreover, multi-speaker augmentation using synthetic data is shown to enhance prosodic coverage without degradation of primary speaker quality, suggesting that controlled synthetic data supplementation is a practical solution to resource-scarce scenarios.

The theoretical implications extend beyond Arabic: cepstral measures and adversarial loss architectures may generalize as standard tools for low-resource, high-variation language TTS, guiding objective model selection and optimization. Practically, these findings underscore the feasibility of deploying compact, adversarially regularized TTS systems in production or on-device environments, as well as the importance of data diversity in achieving robust prosodic synthesis.

Further research is warranted on adaptive pitch/duration loss weighting, variance-preserving predictors, in-domain data augmentation, and efficient model variants for mobile applications.

Conclusion

This work establishes a reproducible, transparent Arabic TTS baseline using FastPitch, introduces principled cepstral metrics for oversmoothing, and demonstrates the efficacy of lightweight adversarial regularization. The methodology maintains robustness under multi-speaker augmentation, sets new standards for objective TTS evaluation, and provides practical recipes for model deployment and further extension. The contributions represent a substantial step forward for Arabic TTS, with broader relevance to global low-resourced TTS research and implementation.