Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition (2310.08132v1)

Published 12 Oct 2023 in cs.CL, cs.SD, and eess.AS

Abstract: Synthetic data generated by text-to-speech (TTS) systems can be used to improve automatic speech recognition (ASR) systems in low-resource or domain mismatch tasks. It has been shown that TTS-generated outputs still do not have the same qualities as real data. In this work we focus on the temporal structure of synthetic data and its relation to ASR training. By using a novel oracle setup we show how much the degradation of synthetic data quality is influenced by duration modeling in non-autoregressive (NAR) TTS. To get reference phoneme durations we use two common alignment methods, a hidden Markov Gaussian-mixture model (HMM-GMM) aligner and a neural connectionist temporal classification (CTC) aligner. Using a simple algorithm based on random walks we shift phoneme duration distributions of the TTS system closer to real durations, resulting in an improvement of an ASR system using synthetic data in a semi-supervised setting.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. “Semi-supervised sequence-to-sequence ASR using unpaired speech and text,” in Interspeech 2019. September 2019, ISCA.
  2. “Speech recognition with augmented synthesized speech,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), December 2019.
  3. “Generating synthetic audio data for attention-based speech recognition systems,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). January 2020, number 9052899, pp. 7069–7073, IEEE.
  4. “You do not need more data: Improving end-to-end speech recognition by text-to-speech data augmentation,” 2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), October 2020.
  5. “Eat: Enhanced ASR-TTS for self-supervised speech recognition,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). June 2021, IEEE.
  6. “Using Synthetic Audio to Improve the Recognition of Out-of-Vocabulary Words in End-to-End Asr Systems,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). June 2021, IEEE.
  7. “Comparing the benefit of synthetic training data for various automatic speech recognition architectures,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2021, pp. 788–795.
  8. “Exploring neural transducers for end-to-end speech recognition,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2017, pp. 206–213.
  9. “Listen, attend and spell: A neural network for large vocabulary conversational speech recognition,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 4960–4964.
  10. “Natural TTS synthesis by conditioning wavenet on MEL spectrogram predictions,” in ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). April 2018, IEEE.
  11. “Data augmentation for asr using tts via a discrete representation,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), December 2021, pp. 68–75.
  12. “FastSpeech 2: Fast and High-Quality End-to-End Text to Speech,” in International Conference on Learning Representations (ICLR), December 2021.
  13. Adrian Łańcucki, “Fastpitch: Parallel text-to-speech with pitch prediction,” in ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6588–6592.
  14. “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proceedings of the 38th International Conference on Machine Learning, Marina Meila and Tong Zhang, Eds. 18–24 Jul 2021, vol. 139 of Proceedings of Machine Learning Research, pp. 5530–5540, PMLR.
  15. “SYNT++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2022.
  16. “Evaluating and reducing the distance between synthetic and real speech distributions,” ArXiv, vol. abs/2211.16049, November 2022.
  17. “Evaluating Speech–Phoneme Alignment and its Impact on Neural Text-To-Speech Synthesis,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1 2023, number 10094560, pp. 1–5, IEEE.
  18. “Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi,” in Interspeech 2017. August 2017, ISCA.
  19. “VRAIN-UPV MLLP’s system for the Blizzard Challenge 2021,” Festvox Blizzard Challenge 2021, October 2021.
  20. “Librispeech: An asr corpus based on public domain audio books.,” in ICASSP. 2015, pp. 5206–5210, IEEE.
  21. “RASR/NN: The RWTH neural network toolkit for speech recognition,” in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). May 2014, IEEE.
  22. “Returnn: The RWTH extensible training framework for universal recurrent neural networks,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017, New Orleans, LA, USA, March 5-9, 2017, pp. 5345–5349.
  23. “Sisyphus, a workflow manager designed for machine translation and automatic speech recognition,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 - November 4, 2018, pp. 84–89.
  24. “Non-attentive tacotron: Robust and controllable neural TTS synthesis including unsupervised duration modeling,” ArXiv, vol. abs/2010.04301v3, October 2020.
  25. “Long short-term memory,” Neural computation, vol. 9, no. 8, 1997.
  26. “Speech synthesis from short-time fourier transform magnitude and its application to speech processing,” in ICASSP ’84, San Diego, California, USA, March 19-21, 1984, pp. 61–64.
  27. “Joint-sequence models for grapheme-to-phoneme conversion,” Speech Communication, vol. 50, no. 5, pp. 434–451, May 2008.
  28. Steve J Young, “The general use of tying in phoneme-based HMM speech recognisers,” in ICASSP, 1992.
  29. M.J.F. Gales, “Maximum likelihood linear transformations for HMM-based speech recognition,” Computer Speech & Language, vol. 12, no. 2, pp. 75–98, April 1998.
  30. “Improved training of end-to-end attention models for speech recognition,” in Interspeech 2018. September 2018, ISCA.
  31. “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proc. Interspeech 2020, 2020, pp. 5036–5040.
  32. “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,” in Proc. Interspeech 2019, 2019, pp. 2613–2617.
  33. “Language Modeling with Deep Transformers,” in Proc. Interspeech 2019, 2019, pp. 3905–3909.
  34. R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” in Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing, 1993, vol. 1, pp. 125–128 vol.1.
  35. “Automos: Learning a non-intrusive assessor of naturalness-of-speech,” in NIPS 2016 End-to-end Learning for Speech and Audio Processing Workshop, 2016.
  36. “The VoiceMOS Challenge 2022,” in Proc. Interspeech 2022, 2022, pp. 4536–4540.
  37. “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Aug. 2016, pp. 1715–1725, Association for Computational Linguistics.
  38. “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun, Eds., 2015.
  39. “LibriTTS: A corpus derived from librispeech for text-to-speech,” in Interspeech 2019. September 2019, ISCA.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Nick Rossenbach (9 papers)
  2. Benedikt Hilmes (6 papers)
  3. Ralf Schlüter (73 papers)
Citations (2)
View on Semantic Scholar