Reducing Dependence on the Guiding Streaming ASR Model

Reduce the dependence of StreamAlign’s alignment quality on its guiding streaming automatic speech recognition model while preserving accurate streaming speech-text alignment and reconstruction.

Background

StreamAlign obtains transcription guidance from a frozen, pretrained streaming word-level automatic speech recognition model and decomposes the resulting word hypotheses into characters for RNN-Transducer alignment. Consequently, alignment quality remains tied to the recognizer’s outputs.

The paper finds that recognition errors are not copied deterministically into the resulting speech units, but it does not eliminate the underlying dependence on the guiding recognizer. The authors therefore explicitly identify reducing this dependence as an open problem.