Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

More than words: Advancements and challenges in speech recognition for singing (2403.09298v1)

Published 14 Mar 2024 in cs.SD, cs.CL, cs.IR, cs.LG, and eess.AS

Abstract: This paper addresses the challenges and advancements in speech recognition for singing, a domain distinctly different from standard speech recognition. Singing encompasses unique challenges, including extensive pitch variations, diverse vocal styles, and background music interference. We explore key areas such as phoneme recognition, language identification in songs, keyword spotting, and full lyrics transcription. I will describe some of my own experiences when performing research on these tasks just as they were starting to gain traction, but will also show how recent developments in deep learning and large-scale datasets have propelled progress in this field. My goal is to illuminate the complexities of applying speech recognition to singing, evaluate current capabilities, and outline future research directions.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. IEEE Signal Processing Magazine, 36(1), pp. 82–94, 2018.
  2. Hansen, J. K.: Recognition of phonemes in a-cappella recordings using temporal patterns and mel frequency cepstral coefficients. In Sound and Music Computing Conference (SMC). 2012.
  3. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 20(1), pp. 200–210, 2012.
  4. Kruspe, A. M.: Bootstrapping a system for phoneme recognition and keyword spotting in unaccompanied singing. In 17th International Society for Music Information Retrieval Conference (ISMIR). New York, NY, USA, 2016a.
  5. Roa Dabike, G. and J. Barker: Automatic lyric transcription from karaoke vocal tracks: Resources and a baseline system. In Interspeech. 2019. doi:10.21437/Interspeech.2019-2378.
  6. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, pp. 2382–2395, 2021. doi:10.1109/TASLP.2021.3091817.
  7. CoRR, abs/2110.05580, 2021. URL https://arxiv.org/abs/2110.05580. 2110.05580.
  8. In 19th International Society for Music Information Retrieval Conference. Paris, France, 2018. Hal-02019115.
  9. Transactions of the International Society for Music Information Retrieval, 3(1), pp. 55–67, 2020. doi:10.5334/tismir.30.
  10. Journal of Voice, 14(2), pp. 287–298, 2000.
  11. In Interspeech. 2003.
  12. In International Society for Music Information Retrieval Conference (ISMIR). 2005.
  13. Mesaros, A. and T. Virtanen: Automatic recognition of lyrics in singing. EURASIP Journal on Audio, Speech, and Music Processing, 2010, pp. 1–11, 2010a.
  14. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2014.
  15. In NASA STI/Recon Technical Report N, vol. 93. National Institute of Standards and Technology (NIST), 1993.
  16. Kruspe, A. M.: Training phoneme models for singing with ”songified” speech data. In 16th International Society for Music Information Retrieval Conference (ISMIR). Malaga, Spain, 2015a.
  17. ArXiv, abs/2109.07940, 2021. URL https://api.semanticscholar.org/CorpusID:237532132.
  18. Tsai, W.-H. and H.-M. Wang: Towards automatic identification of singing language in popular music recordings. In International Society for Music Information Retrieval Conference (ISMIR). 2004.
  19. In International Society for Music Information Retrieval Conference (ISMIR). 2006.
  20. Mehrabani, M. and J. H. L. Hansen: Language identification for singing. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2011.
  21. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2011.
  22. In Proc. of the AES Conference on Semantic Audio. London, UK, 2014.
  23. Kruspe, A. M.: Improving singing language identification through i-vector extraction. In Proc. of the 17th Int. Conference on Digital Audio Effects (DAFx-14). Erlangen, Germany, 2014a.
  24. Kruspe, A. M.: Phonotactic language identification for singing. In Interspeech. San Francisco, CA, USA, 2016b.
  25. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 271–275. 2021. doi:10.1109/ICASSP39728.2021.9414203.
  26. In International Society for Music Information Retrieval Conference. 2021.
  27. In 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 399–404. IEEE, 2020.
  28. In International Society for Music Information Retrieval Conference (ISMIR). 2008.
  29. Nakano, T. and M. Goto: Lyriclistplayer: A consecutive-query-by-playback interface for retrieving similar word sequences from different song lyrics. In Sound and Music Computing Conference (SMC). 2016.
  30. In 3rd International Workshop on Cognitive Information Processing (CIP). 2012.
  31. In International Society for Music Information Retrieval Conference (ISMIR). 2015.
  32. Kruspe, A. M.: Keyword spotting in a-capella singing. In 15th International Society for Music Information Retrieval Conference (ISMIR). Taipei, Taiwan, 2014b.
  33. Kruspe, A. M.: Keyword spotting in singing with duration-modeled hmms. In European Signal Processing Conference (EUSIPCO). Nice, France, 2015b.
  34. In International Computer Music Conference (ICMC). 1999.
  35. In ACM International Conference on Multimedia. 2004.
  36. In SPIE Multimedia Computing and Networking. 2006.
  37. In Sound and Music Computing Conference (SMC). 2010.
  38. Multimedia Systems, 12(4-5), pp. 307–323, 2007.
  39. Lee, K. and M. Cremer: Segmentation-based lyrics-audio alignment using dynamic programming. In International Society for Music Information Retrieval Conference (ISMIR). 2008.
  40. Mesaros, A. and T. Virtanen: Automatic alignment of music audio and lyrics. In International Conference on Digital Audio Effects (DAFx-08). 2008.
  41. In Interspeech. 2015.
  42. In International Society for Music Information Retrieval Conference (ISMIR). 2016.
  43. In 18th International Society for Music Information Retrieval Conference (ISMIR) (MIREX submission). Suzhou, China, 2017.
  44. 2019. 1906.10369.
  45. 2019. 1902.06797.
  46. In International Society for Music Information Retrieval Conference (ISMIR). Montreal, Canada, 2020. URL https://hal.science/hal-02996940.
  47. 2021a. 2102.09202.
  48. In International Society for Music Information Retrieval Conference (ISMIR). 2010.
  49. Mesaros, A. and T. Virtanen: Recognition of phonemes and words in singing. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2010b.
  50. Kruspe, A. M.: Retrieval of textual song lyrics from sung inputs. In Interspeech. San Francisco, CA, USA, 2016c.
  51. Kruspe, A. M. and M. Goto: Retrieval of song lyrics from sung queries. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Calgary, Canada, 2018.
  52. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5749–5753. 2018. doi:10.1109/ICASSP.2018.8462247.
  53. 2020. 2007.06486.
  54. 2021b. 2108.02625.
  55. 2022. 2204.03307.
  56. 2022. 2207.09747.
  57. 2023. 2306.17103.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now