Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition (1507.06947v1)

Abstract: We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained phone models initialized with connectionist temporal classification (CTC). In this paper, we present techniques that further improve performance of LSTM RNN acoustic models for large vocabulary speech recognition. We show that frame stacking and reduced frame rate lead to more accurate models and faster decoding. CD phone modeling leads to further improvements. We also present initial results for LSTM RNN models outputting words directly.

Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition

The paper "Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition" by Haşim Sak, Andrew Senior, Kanishka Rao, and Françoise Beaufays explores advanced methodologies to enhance the performance of LSTM RNNs in speech recognition. The research builds on previous findings that LSTM RNNs outperform feed-forward DNNs and evaluates different modeling techniques that promise accuracy improvements and computational efficiency.

Key Methodologies and Findings

The paper introduces several innovative techniques aimed at improving speech recognition models, notably through the use of LSTM RNNs combined with connectionist temporal classification (CTC) training. The authors leverage sequence-trained phone models initialized with CTC to reach levels of performance that compete with state-of-the-art sequence-trained context-dependent HMM models.

1. Frame Stacking and Reduced Frame Rate: These techniques lead to the development of more accurate models and expedited decoding processes. Instead of processing each individual frame, multiple frames are stacked and processed collectively, allowing a significant reduction in computational load and complexity, while maintaining the integrity of acoustic information.
2. Context-Dependent (CD) Phone Modeling: By incorporating CD phone units, notable improvements in recognition accuracy are observed. The paper demonstrates that CD phone models offer an 8% relative improvement in recognition accuracy over conventional LSTM RNN models.
3. Training and Evaluation: The paper uses the state-level minimum Bayes risk (sMBR) sequence discriminative training criterion to enhance the models initially trained with CTC or cross-entropy loss. The sMBR approach consistently improved the performance across different model initializations, indicating its effectiveness in minimizing word error rates (WER).
4. Word-Level Acoustic Modeling: Initial investigations reveal the potential of LSTM models to directly output words, bypassing the phoneme-level representation. Although the model exhibits promising accuracy for medium-sized vocabularies without a LLM, further research is suggested to refine this approach for larger vocabulary tasks.

Experimental Results

The evaluation carried out on a large dataset of transcribed voice search traffic demonstrated significant performance improvements with the new methods:

• The CTC CD phone models showed superior performance among the evaluated models, reducing WER by 8% relative in unidirectional models and 4% in bidirectional models post-sMBR training.
• Unidirectional models presented marginally better results compared to bidirectional models, particularly in terms of computation speed and applicability to low-latency applications.

Implications and Future Directions

The research illustrates the potential of LSTM RNNs bused with novel training techniques in improving the speech recognition landscape. The reduced frame processing time and accurate modeling achieved in this paper offer practical advantages for real-time applications where computational resources are constrained. Furthermore, the incorporation of word-level acoustic modeling paves the way for new directions in language processing, suggesting the feasibility of end-to-end recognition systems not reliant on LLMs.

Future work may focus on optimizing the trade-off between model accuracy and computational efficiency further and expanding the vocabulary breadth for direct word model outputs. These advances would bear implications for broader AI applications where real-time interaction is paramount. The exploration of other training paradigms and model architectures could also augment the current understanding and application of LSTM RNNs in various sequence learning tasks.