Non-intrusive speech intelligibility prediction using automatic speech recognition derived measures (2010.08574v2)

Abstract: The estimation of speech intelligibility is still far from being a solved problem. Especially one aspect is problematic: most of the standard models require a clean reference signal in order to estimate intelligibility. This is an issue of some significance, as a reference signal is often unavailable in practice. In this work, therefore a non-intrusive speech intelligibility estimation framework is presented. In it, human listeners' performance in keyword recognition tasks is predicted using intelligibility measures that are derived from models trained for automatic speech recognition (ASR). One such ASR-based and one signal-based measure are combined into a full framework, the proposed NO-Reference Intelligibility (Nori) estimator, which is evaluated in predicting the performance of both normal-hearing and hearing-impaired listeners in multiple noise conditions. It is shown that the Nori framework even outperforms the widely used reference-based (or intrusive) short-term objective intelligibility (STOI) measure in most considered scenarios, while being applicable in fully blind scenarios with no reference signal or transcription, creating perspectives for online and personalized optimization of speech enhancement systems.