Foundation Models in Augmentative and Alternative Communication: Opportunities and Challenges (2401.08866v1)
Abstract: Augmentative and Alternative Communication (AAC) are essential techniques that help people with communication disabilities. AAC demonstrates its transformative power by replacing spoken language with symbol sequences. However, to unlock its full potential, AAC materials must adhere to specific characteristics, placing the onus on educators to create custom-tailored materials and symbols. This paper introduces AMBRA (Pervasive and Personalized Augmentative and Alternative Communication based on Federated Learning and Generative AI), an open platform that aims to leverage the capabilities of foundation models to tackle many AAC issues, opening new opportunities (but also challenges) for AI-enhanced AAC. We thus present a compelling vision--a roadmap towards a more inclusive society. By leveraging the capabilities of modern technologies, we aspire to not only transform AAC but also guide the way toward a world where communication knows no bounds.
- “Language Models Are Few-Shot Learners” In Proc. of NIPS, 2020
- Louise Cummings “Communication disorders: A complex population in healthcare” In Language and Health 1.2, 2023, pp. 12–19 DOI: https://doi.org/10.1016/j.laheal.2023.06.005
- “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, 2019 arXiv:1810.04805 [cs.CL]
- “Augmentative and Alternative Communication on Tablet to Help Persons with Severe Disabilities” In Proc. of CHItaly, 2013
- “Access to AAC: Present, past, and future” In Augmentative and Alternative Communication 23.3 Taylor & Francis, 2007, pp. 243–257
- “Artificial Intelligence in Augmentative and Alternative Communication Systems: A Literature-Based Assessment and Implications of Different Conversation Phases and Contexts” In Proc. of ECIS, 2023
- “Pretrained Language Models for Text Generation: A Survey”, 2021 arXiv:2105.10311 [cs.CL]
- “Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing” In ACM Comput. Surv. 55.9, 2023
- “Recurrent neural network based language model” In Proc. of INTERSPEECH, 2010
- OpenAI “DALL·E (Dec 1 version)” Accessed: Dec. 16, 2023, [Large language model], 2023 URL: https://chat.openai.com/?model=gpt-4
- Murray Shanahan “Talking About Large Language Models”, 2023 arXiv:2212.03551 [cs.CL]
- “LLaMA: Open and Efficient Foundation Language Models”, 2023 arXiv:2302.13971 [cs.CL]
- “The Less I Type, the Better: How AI Language Models Can Enhance or Impede Communication for AAC Users” In Proc. of CHI, 2023 DOI: 10.1145/3544548.3581560
- “Finetuned Language Models Are Zero-Shot Learners”, 2022 arXiv:2109.01652 [cs.CL]
- Widgit Software “Widgit Symbols”, 2023 URL: www.widgit.com
- “A Survey of Large Language Models”, 2023 arXiv:2303.18223 [cs.CL]
- “Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis”, 2023 arXiv:2304.04675 [cs.CL]