Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

"We do use it, but not how hearing people think": How the Deaf and Hard of Hearing Community Uses Large Language Model Tools (2410.21358v3)

Published 28 Oct 2024 in cs.HC

Abstract: Generative AI tools, particularly those utilizing LLMs, are increasingly used in everyday contexts. While these tools enhance productivity and accessibility, little is known about how Deaf and Hard of Hearing (DHH) individuals engage with them or the challenges they face when using them. This paper presents a mixed-method study exploring how the DHH community uses Text AI tools like ChatGPT to reduce communication barriers and enhance information access. We surveyed 80 DHH participants and conducted interviews with 11 participants. Our findings reveal important benefits, such as eased communication and bridging Deaf and hearing cultures, alongside challenges like lack of American Sign Language (ASL) support and Deaf cultural understanding. We highlight unique usage patterns, propose inclusive design recommendations, and outline future research directions to improve Text AI accessibility for the DHH community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Artificial intelligence (AI) for web accessibility: Is conformance evaluation a way forward?. In Proceedings of the 15th international web for all conference. 1–4.
  2. Customization of closed captions via large language models. In International Conference on Computers Helping People with Special Needs. Springer, 50–58.
  3. Yael Bat-Chava. 2000. Diversity of deaf identities. American annals of the deaf (2000), 420–428.
  4. H-Dirksen L Bauman and Joseph J Murray. 2014. Deaf gain: Raising the stakes for human diversity. U of Minnesota Press.
  5. A study on automatic speech recognition. Journal of Information Technology Review 10, 3 (2019), 77–85.
  6. Sign language recognition, generation, and translation: An interdisciplinary perspective. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility. 16–31.
  7. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
  8. Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589–597.
  9. Neural sign language translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7784–7793.
  10. Sign language transformers: Joint end-to-end sign language recognition and translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10023–10033.
  11. Improving DHH students’ grammar through an individualized software program. Journal of Deaf Studies and Deaf Education 16, 4 (2011), 437–457.
  12. Joanna E Cannon and Susannah Kirby. 2013. Grammar structures and deaf and hard of hearing students: A review of past performance and a report of new findings. American annals of the deaf 158, 3 (2013), 292–310.
  13. Joanna E Cannon and John L Luckner. 2016. Increasing cultural and linguistic diversity in deaf education teacher preparation programs. American Annals of the Deaf 161, 1 (2016), 89–103.
  14. ”Real Learner Data Matters” Exploring the Design of LLM-Powered Question Generation for Deaf and Hard of Hearing Learners. arXiv:2410.00194 [cs.HC] https://arxiv.org/abs/2410.00194
  15. Adrienne Colella and Dianna L Stone. 2013. Workplace discrimination toward persons with disabilities: A call for some new research directions. In Discrimination at work. Psychology Press, 227–253.
  16. Kata Csizér and Edit H Kontra. 2020. Foreign language learning characteristics of deaf and severely hard-of-hearing students. The Modern Language Journal 104, 1 (2020), 233–249.
  17. ”That comes with a huge career cost:” Understanding Collaborative Ideation Experiences of Disabled Professionals. Proceedings of the ACM on Human-Computer Interaction 8, CSCW1 (2024), 1–28.
  18. ASL citizen: a community-sourced dataset for advancing isolated sign language recognition. Advances in Neural Information Processing Systems 36 (2024).
  19. Systemic Biases in Sign Language AI Research: A Deaf-Led Call to Reevaluate Research Agendas. arXiv preprint arXiv:2403.02563 (2024).
  20. Ewa Domagała-Zyśk and Anna Podlewska. 2019. Strategies of oral communication of deaf and hard-of-hearing (D/HH) non-native English users. European Journal of Special Needs Education 34, 2 (2019), 156–171.
  21. Claire Edwards and Gill Harold. 2014. DeafSpace and the principles of universal design. Disability and Rehabilitation 36, 16 (2014), 1350–1359.
  22. Generative ai. Business & Information Systems Engineering 66, 1 (2024), 111–126.
  23. Sally T Freeman. 1989. Cultural and linguistic bias in mental health evaluations of deaf people. Rehabilitation Psychology 34, 1 (1989), 51.
  24. The Future of Urban Accessibility: The Role of AI. (2024).
  25. Neil S Glickman and Wyatte C Hall. 2018. Language deprivation and deaf mental health. Routledge.
  26. Blaine Goss. 2003. Hearing from the deaf culture. Intercultural communication studies 12, 2 (2003), 1–17.
  27. The impact of assistive technology on communication quality between deaf and hearing individuals. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. 669–682.
  28. Deaf children need language, not (just) speech. First Language 39, 4 (2019), 367–395.
  29. Wyatte C Hall. 2017. What you don’t know can hurt you: The risk of language deprivation by impairing sign language development in deaf children. Maternal and child health journal 21, 5 (2017), 961–965.
  30. Tracie Heskett. 2001. Using the Six Trait Writing Model. Teacher Created Resources.
  31. Iain Hutchison. 2007. Oralism: A sign of the times? The contest for deaf communication in education provision in late nineteenth-century Scotland. European Review of History—Revue européenne d’Histoire 14, 4 (2007), 481–501.
  32. Ethical implications of text generation in the age of artificial intelligence. Business Ethics, the Environment & Responsibility 32, 1 (2023), 201–210.
  33. Hamid Reza Vaezi Joze and Oscar Koller. 2018. Ms-asl: A large-scale data set and benchmark for understanding american sign language. arXiv preprint arXiv:1812.01053 (2018).
  34. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. International journal of epidemiology 31, 1 (2002), 115–123.
  35. The Sem-Lex Benchmark: Modeling ASL Signs and Their Phonemes. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility. 1–10.
  36. Edit H Kontra. 2017. The foreign-language learning situation of Deaf adults: An overview. Journal of Adult Learning, Knowledge and Innovation 1, 1 (2017), 35–42.
  37. Paddy Ladd. 2005. Deafhood: A concept stressing possibilities, not deficits. Scandinavian Journal of Public Health 33, 66_suppl (2005), 12–17.
  38. An introduction to teaching with text generation technologies. TextGenEd: Teaching with text generation technologies (2023).
  39. Yiwen Li. 2024. Usability of ChatGPT in Second Language Acquisition: Capabilities, Effectiveness, Applications, Challenges, and Solutions. Studies in Applied Linguistics and TESOL 24, 1 (2024).
  40. Discrimination and deaf adolescents’ subjective well-being: The role of deaf identity. The Journal of Deaf Studies and Deaf Education 27, 4 (2022), 399–407.
  41. Social app accessibility for deaf signers. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–31.
  42. Anticipate and adjust: Cultivating access in human-centered methods. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18.
  43. “They only care to show us the wheelchair”: disability representation in text-to-image AI models. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–23.
  44. Social, environmental, and technical: Factors at play in the current use and future design of small-group captioning. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–25.
  45. Brian T McMahon. 2012. An overview of workplace discrimination and disability. Journal of Vocational Rehabilitation 36, 3 (2012), 135–139.
  46. Understanding Personalized Accessibility through Teachable AI: Designing and Evaluating Find My Things for People who are Blind or Low Vision. In Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility. 1–12.
  47. A survey on the animation of signing avatars: From sign representation to utterance synthesis. Computers & Graphics 92 (2020), 76–98.
  48. Noel O’Connell. 2022. “Opportunity blocked”: Deaf people, employment and the sociology of audism. Humanity & society 46, 2 (2022), 336–358.
  49. Austin Pack and Jeffrey Maloney. 2023. Potential affordances of generative ai in language education: Demonstrations and an evaluative framework. Teaching English with Technology 23, 2 (2023), 4–24.
  50. Artificial intelligence technologies for sign language. Sensors 21, 17 (2021), 5843.
  51. Ila Parasnis. 1998. Cultural and language diversity and the deaf experience. Cambridge University Press.
  52. Sen Qi and Ross E Mitchell. 2012. Large-scale academic achievement testing of deaf and hard-of-hearing students: Past, present, and future. Journal of deaf studies and deaf education 17, 1 (2012), 1–18.
  53. Language learning practices with deaf children. ERIC.
  54. Language learning with restricted input: Case studies of two hearing children of deaf parents. Applied Psycholinguistics 2, 1 (1981), 33–54.
  55. Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy. Nursing & health sciences 12, 3 (2010), 369–374.
  56. Sven Schild and Constance J Dalenberg. 2012. Trauma exposure and traumatic symptoms in deaf adults. Psychological trauma: Theory, research, practice, and policy 4, 1 (2012), 117.
  57. Sven Schild and Constance J Dalenberg. 2016. Information deprivation trauma: Definition, assessment, and interventions. Journal of aggression, maltreatment & trauma 25, 8 (2016), 873–889.
  58. A Call for a Diversity of Perspectives in Deaf Education Research. American Annals of the Deaf 166, 1 (2021), 49–61.
  59. Per Koren Solvang and Hilde Haualand. 2014. Accessibility and diversity: Deaf space in action. Scandinavian Journal of Disability Research 16, 1 (2014), 1–13.
  60. PopSign ASL v1. 0: an isolated american sign language dataset collected via smartphones. Advances in Neural Information Processing Systems 36 (2024).
  61. William C Stokoe Jr. 2005. Sign language structure: An outline of the visual communication systems of the American deaf. Journal of deaf studies and deaf education 10, 1 (2005), 3–37.
  62. Michael Strong. 1988. Language learning and deafness. Cambridge University Press.
  63. Why American sign language gloss must matter. American annals of the deaf 161, 5 (2017), 540–551.
  64. M Virginia Swisher. 1989. The language-learning situation of deaf students. Tesol Quarterly 23, 2 (1989), 239–257.
  65. Disability-first dataset creation: lessons from constructing a dataset for teachable object recognition with blind and low vision data collectors. In Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility. 1–12.
  66. Clayton Valli. 2000. Linguistics of American Sign Language: An Introduction. Gallaudet University Press.
  67. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. 214–229.
  68. A description of ASL features in writing. Ampersand 1 (2014), 19–27.

Summary

  • The paper demonstrates that text-based AI tools enhance communication by improving proofreading, grammar correction, and vocabulary support for DHH individuals.
  • It reveals that AI serves as a cultural mediator, helping bridge communication gaps between Deaf and hearing communities by adjusting tone and context.
  • The study employs surveys (N=80) and interviews (N=11) to identify challenges like prompt complexity and the need for culturally inclusive AI features.

Analyzing the Use of LLMs by the Deaf and Hard of Hearing Community

The paper "We do use it, but not how hearing people think": How the Deaf and Hard of Hearing Community Uses LLM Tools," presents an in-depth examination of how individuals within the Deaf and Hard of Hearing (DHH) community utilize text-based generative AI tools such as ChatGPT. The paper conducted by Huffman et al. applies a mixed-method approach, combining a survey and complementary interviews to investigate both the benefits and barriers associated with the use of these tools by DHH users.

Survey and Methods Overview

The paper surveyed 80 DHH individuals and conducted interviews with 11 participants to explore key research questions regarding the interaction of the DHH community with text-based generative AI tools. The survey focused on multiple aspects: frequency of use, types of tasks for which AI is employed, its role in reducing anxiety and discrimination, and challenges faced by the users. The structured methodology incorporated instruments available in both American Sign Language (ASL) and English to accommodate linguistic diversity within the community.

Key Findings: Benefits and Usage

  1. Communication Confidence: The paper highlights that many participants found text-based AI tools beneficial in enhancing their English language confidence. This aligns with the challenges faced by the DHH community, which often encounters linguistic and cultural barriers in predominantly hearing societies. Many participants used AI for proof-reading, grammar correction, and vocabulary enhancement, which in turn improved their confidence in written communication.
  2. Cultural Mediation: AI tools were seen as effective cultural mediators, bridging gaps between Deaf and hearing worlds. Participants used AI to adjust the tone of their communication, presenting themselves in a more culturally intuitive manner to hearing individuals. The tools also assisted in communicating complex emotional and cultural contexts, reducing misunderstandings stemming from differences between Deaf and hearing communication styles.
  3. Information Retrieval and Fact-Checking: A prevalent use of AI involved quick and effective information retrieval and fact-checking. Participants preferred AI for its interactive capabilities, which allowed for iterative querying that refined their understanding and facilitated knowledge acquisition, reducing the burden traditionally associated with literacy disparities.

Concerns and Challenges

  1. Prompt Complexity: A primary challenge noted was the reliance on English proficiency to effectively craft prompts. Participants reported difficulties in constructing suitable queries, which is symptomatic of broader educational and linguistic divides faced by the DHH community.
  2. Cultural and Functional Limitations: While AI tools offer communication support, their current iterations lack a nuanced understanding of Deaf culture and context. This limitation is rooted in the training datasets, which predominantly reflect hearing culture biases. Users also pointed to a lack of specific features, like ASL support, which would significantly improve accessibility.
  3. Trust and Privacy Issues: Some participants expressed concerns about the reliability and privacy of AI tools, underscoring a need for more trust-building measures in the design and implementation of AI systems for diverse communities.

Recommendations and Future Directions

The paper underscores the importance of culturally and linguistically inclusive design in AI technologies. Recommendations include integrating ASL input and output capabilities, and culturally aware datasets in training models, to offer more tailored support to the DHH community. Furthermore, employing educational programs to assist DHH users in optimizing AI interaction could mitigate current limitations of prompt complexity and user experience.

Huffman et al.'s research emphasizes the need for future research to address AI bias and enhance accessibility, ultimately promoting a more inclusive technological landscape. The paper acts as a beacon for ongoing improvements in AI accessibility, highlighting the unique requirements of historically marginalized user communities.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets