Translation of recognized speech attributes into accessible representations

Determine how reliably recognized emotion, tone, and delivery attributes from generated speech can be translated into written English or American Sign Language representations that are readily understandable to Deaf and hard-of-hearing users.

Background

The proposed AI-based verification approach would use automatic speech recognition to identify emotion, tone, and delivery in generated speech. However, recognition alone does not solve the accessibility problem: the recognized attributes must be rendered in a form that users can understand without relying on auditory perception. The paper explicitly leaves unresolved how these attributes should be mapped into written English or American Sign Language in a clear and usable way.

References

Even if ASR were capable of recognizing these attributes reliably, there are open questions as to how these could be translated into written English or ASL in a way that is readily understandable.

— Seeing the Voice, Preserving the Self: A Participatory Design Approach to Deaf-Centric Text-to-Speech  (2609.10199 - Atemnkeng et al., 9 Sep 2026) in Section 5, “Technical Requirements for Implementing the Designs,” subsection “Supporting AI-based Verification”