AIx Speed: Playback Speed Optimization Using Listening Comprehension of Speech Recognition Models
Abstract: Since humans can listen to audio and watch videos at faster speeds than actually observed, we often listen to or watch these pieces of content at higher playback speeds to increase the time efficiency of content comprehension. To further utilize this capability, systems that automatically adjust the playback speed according to the user's condition and the type of content to assist in more efficient comprehension of time-series content have been developed. However, there is still room for these systems to further extend human speed-listening ability by generating speech with playback speed optimized for even finer time units and providing it to humans. In this study, we determine whether humans can hear the optimized speech and propose a system that automatically adjusts playback speed at units as small as phonemes while ensuring speech intelligibility. The system uses the speech recognizer score as a proxy for how well a human can hear a certain unit of speech and maximizes the speech playback speed to the extent that a human can hear. This method can be used to produce fast but intelligible speech. In the evaluation experiment, we compared the speech played back at a constant fast speed and the flexibly speed-up speech generated by the proposed method in a blind test and confirmed that the proposed method produced speech that was easier to listen to.
- Video summarization using deep neural networks: A survey. Proc. IEEE 109, 11 (2021), 1838–1863.
- Wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. In Advances in Neural Information Processing Systems.
- Zx Bai and Xiao-Lei Zhang. 2021. Speaker recognition based on deep learning: An overview. Neural Networks (2021).
- SmartPlayer: User-Centric Video Fast-Forwarding. In Proc. of the SIGCHI Conference on Human Factors in Computing Systems.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186.
- Songshuang Duan and Xiaoqian Chen. 2019. Why College Students Watch Streaming Drama at Higher Playback Speed: The Uses and Gratifications Perspective. In International Joint Conference on Information, Media and Engineering.
- Automatic speech recognition predicts speech intelligibility and comprehension for listeners with simulated age-related hearing loss. Journal of Speech, Language, and Hearing Research 60, 9 (2017), 2394–2405.
- Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Nets. In ICML ’06: Proceedings of the International Conference on Machine Learning.
- EgoScanning: Quickly Scanning First-Person Videos with Egocentric Elastic Timelines. In Proc. ACM Conference on Human Factors in Computing Systems.
- HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units. (2021).
- Wenyu Jiang and H. Schulzrinne. 2002. Speech recognition performance as an effective perceived quality predictor. In IEEE 2002 Tenth IEEE International Workshop on Quality of Service (Cat. No.02EX564). 269–275.
- A Survey of Speaker Recognition: Fundamental Theories, Recognition Methods and Opportunities. IEEE Access 9 (2021), 79236–79263.
- Efficient Video Viewing System for Racquet Sports with Automatic Summarization Focusing on Rally Scenes. In ACM SIGGRAPH 2014 Posters.
- Dynamic Object Scanning: Object-Based Elastic Timeline for Quickly Browsing First-Person Videos. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems.
- Kazutaka Kurihara. 2011. CinemaGazer: A System for Watching Video at Very High Speed. In Proc. of the Workshop on Advanced Visual Interfaces AVI.
- Is Faster Better? A Study of Video Playback Speed. In Proc. of the Tenth International Conference on Learning Analytics & Knowledge.
- Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In International Conference on Learning Representations.
- Automatic speech recognition: A survey. Multimedia Tools and Applications (2021).
- Librosa: Audio and music signal analysis in python. In Proceedings of the 14th python in science conference. Citeseer.
- English Speech Database Read by Japanese Learners for CALL System Development.. In LREC. Citeseer.
- Learning in double time: The effect of lecture video speed on immediate and delayed comprehension. Applied Cognitive Psychology 36, 1 (2022), 69–82.
- Toru Nagahama and Yusuke Morita. 2017. Effect Analysis of Playback Speed for Lecture Video Including Instructor Images. International Journal for Educational Media and Technology (2017).
- Laugh at Your Own Pace: Basic Performance Evaluation of Language Learning Assistance by Adjustment of Video Playback Speeds Based on Laughter Detection. In Proc. of the Ninth ACM Conference on Learning @ Scale.
- Librispeech: An ASR corpus based on public domain audio books. In IEEE International Conference on Acoustics, Speech and Signal Processing.
- PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32.
- Automatically Adjusting the Speed of E-Learning Videos. In Proc. of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems.
- Automatic Speech Recognition (ASR) Systems Applied to Pronunciation Assessment of L2 Spanish for Japanese Speakers. Applied Sciences 11, 15 (2021).
- WaveNet: A Generative Model for Raw Audio. In Arxiv.
- Audio Summarization for Podcasts. In 2021 29th European Signal Processing Conference (EUSIPCO). IEEE, 431–435.
- Tacotron: Towards End-to-End Speech Synthesis. In INTERSPEECH. 4006–4010.
- Achieving Human Parity in Conversational Speech Recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing.
- WithYou: Automated Adaptive Speech Tutoring With Context-Dependent Speech Recognition. In Proc. of the 2020 CHI Conference on Human Factors in Computing Systems.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.