VoxHakka: A Dialectally Diverse Multi-speaker Text-to-Speech System for Taiwanese Hakka
Abstract: This paper introduces VoxHakka, a text-to-speech (TTS) system designed for Taiwanese Hakka, a critically under-resourced language spoken in Taiwan. Leveraging the YourTTS framework, VoxHakka achieves high naturalness and accuracy and low real-time factor in speech synthesis while supporting six distinct Hakka dialects. This is achieved by training the model with dialect-specific data, allowing for the generation of speaker-aware Hakka speech. To address the scarcity of publicly available Hakka speech corpora, we employed a cost-effective approach utilizing a web scraping pipeline coupled with automatic speech recognition (ASR)-based data cleaning techniques. This process ensured the acquisition of a high-quality, multi-speaker, multi-dialect dataset suitable for TTS training. Subjective listening tests conducted using comparative mean opinion scores (CMOS) demonstrate that VoxHakka significantly outperforms existing publicly available Hakka TTS systems in terms of pronunciation accuracy, tone correctness, and overall naturalness. This work represents a significant advancement in Hakka language technology and provides a valuable resource for language preservation and revitalization efforts.
- “Transfer learning from speaker verification to multispeaker text-to-speech synthesis,” in Proc. NeurIPS, 2018.
- “YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone,” in Proc. ICML, 2022.
- “Neural codec language models are zero-shot text to speech synthesizers,” in Arxiv preprint arXiv:2301.02111, 2023.
- “XTTS: A massively multilingual zero-shot text-to-speech model,” in Proc. Interspeech, 2024.
- “CLaM-TTS: Improving neural codec language model for zero-shot text-to-speech,” in Proc. ICLR, 2024.
- “The Kaldi speech recognition toolkit,” in Proc. ASRU, 2011.
- “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proc. ICML, 2021.
- “Clova baseline system for the VoxCeleb speaker recognition challenge 2020,” in Arxiv preprint arXiv:2009.14153, 2020.
- “WaveNet: A generative model for raw audio,” in Proc. SSW, 2016.
- “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” in Proc. NeurIPS, 2020.
- “Auto-encoding variational Bayes,” in ICLR, 2014.
- “Decoupled weight decay regularization,” in Proc. ICLR, 2019.
- “Taiwanese Hakka across Taiwan corpus and Formosa speech recognition challenge 2023 - Hakka ASR,” in Proc. O-COCOSDA, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.