Careless Whisper: Speech-to-Text Hallucination Harms (2402.08021v2)

Abstract: Speech-to-text services aim to transcribe input audio as accurately as possible. They increasingly play a role in everyday life, for example in personal voice assistants or in customer-company interactions. We evaluate Open AI's Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023. While many of Whisper's transcriptions were highly accurate, we find that roughly 1\% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio. We thematically analyze the Whisper-hallucinated content, finding that 38\% of hallucinations include explicit harms such as perpetuating violence, making up inaccurate associations, or implying false authority. We then study why hallucinations occur by observing the disparities in hallucination rates between speakers with aphasia (who have a lowered ability to express themselves using speech and voice) and a control group. We find that hallucinations disproportionately occur for individuals who speak with longer shares of non-vocal durations -- a common symptom of aphasia. We call on industry practitioners to ameliorate these language-model-based hallucinations in Whisper, and to raise awareness of potential biases amplified by hallucinations in downstream applications of speech-to-text models.

• The paper reveals that approximately 1% of Whisper transcriptions include hallucinated phrases, with higher occurrence in aphasic speakers.
• It employs logistic regression on 13,140 audio segments to link speech disfluencies and non-vocal pauses to these hallucinations.
• The analysis advocates for design adjustments to reduce harmful outputs, contrasting Whisper’s approach with other speech-to-text systems.

Analyzing Speech-to-Text Hallucination Harms in OpenAI's Whisper

The proliferation of automated speech-to-text systems has marked a significant technological advancement, leveraging deep learning to transcribe audio with increasing accuracy. In this context, OpenAI's Whisper represents a state-of-the-art solution, purporting superior transcription capabilities relative to its industry competitors. However, as this paper by Koenecke et al. reveals, Whisper is not without its limitations and introduces a critical discussion on the phenomenon of 'hallucinations' in speech-to-text transcriptions. These hallucinations are defined as segments of text that do not correspond to any input audio, posing potential risks and implications across various applications.

Experimental Analysis of Whisper Transcriptions

Koenecke et al.'s paper is centered on evaluating the transcription performance of Whisper, focusing specifically on the hallucinations that unexpectedly occur during the transcription process. According to their findings, approximately 1% of transcriptions contain hallucinated phrases, with a notable portion possessing harmful content. The research involved processing 13,140 audio segments through Whisper, sourced from the AphasiaBank repository, which includes audio of aphasia speakers and control speakers, demonstrating significant reliability in identifying hallucination phenomena.

A striking observation is that hallucination occurrences are not uniformly distributed but are more prevalent in segments involving speakers with aphasia. Specifically, 1.7% of aphasia audio files contained hallucinations, surpassing the 1.2% rate among control speakers. This raises substantial concerns about biases, which may exacerbate existing inequalities for individuals with speech disorders, potentially influencing decisions in high-stakes environments such as healthcare or legal settings.

Investigation of Hallucination Content and Causes

The analysis identifies 312 hallucination instances, categorizing them into themes such as violence, demographic stereotypes, false associations, and fraudulent information. This categorization underscores the potential for such hallucinations to perpetuate damaging stereotypes or misinformation. Furthermore, the paper hypothesizes that these hallucinations may result from the fusion of audio and LLMing in Whisper, akin to that seen in generative LLMs like ChatGPT.

The disparities in hallucinations observed suggest that factors such as non-vocal durations within audio segments significantly contribute to hallucination occurrences. Speech disfluencies—extended pauses typical of aphasic speech—appear particularly predictive of hallucination instances, as confirmed by logistic regression analysis. This correlation with non-verbal audio content underscores the need for nuanced considerations in speech-to-text system design and deployment.

Comparative Analysis and Implications

A notable contrast arises when comparing Whisper's performance with other speech-to-text systems, such as those offered by Google and other major tech companies. The absence of hallucinations in the latter systems suggests fundamental differences in modeling approaches, providing a crucial insight into potential avenues for improving Whisper's fidelity and reliability. The paper suggests mitigating strategies, such as calibrating Whisper’s settings to filter out non-essential randomness or implementing user guidance on handling potential hallucinations.

Conclusion and Future Considerations

This paper elevates the discourse on automated speech-to-text systems, emphasizing the necessity of accounting for inaccuracies like hallucinations that can undermine the utility and trustworthiness of AI technologies. The implications are far-reaching, not only highlighting potential biases against users with inherent speech disfluencies but also accentuating the broader ethical considerations in AI deployments. Future research is encouraged to delve further into the intersectionality of hallucination impacts, aiming to refine models for better inclusivity and fairness.

The work by Koenecke et al. presents a compelling narrative that urges industry practitioners to prioritize transparency and ethical responsibility in addressing the intricacies of AI hallucinations. This ongoing dialogue will be crucial in fostering advancements that respect the diverse needs of all users, ensuring the broader societal benefits of AI technologies are equitably realized.