Lessons in co-creation: the inconvenient truths of inclusive sign language technology development (2408.13171v1)

Abstract: In the era of AI-driven language technologies, there is a growing demand for the participation and leadership of deaf communities in sign language technology development, often framed as co-creation. This paper, developed through collaborative and iterative dialogue between the authors with data from informal participant observations, examines the involvement of the European Union of the Deaf in two EU Horizon 2020 projects, EASIER and SignON. These projects aimed to develop mobile translation applications between signed and spoken languages, bringing together predominantly hearing, non-signing technology experts with predominantly hearing sign language academics and organizations representing deaf end users in large multi-partner consortia. While co-creation is sometimes presented as the best or required way to do research or even as emancipatory, it frequently masks systemic issues of power imbalances and tokenism. Drawing from EUD's experiences of these projects, we highlight several inconvenient truths of co-creation, and propose seven lessons for future initiatives: recognizing deaf partners' invisible labour as work, managing expectations about technologies, cripping co-creation processes, exploring alternative methods to mitigate co-creation fatigue, seeking intersectional feedback, ensuring co-creation is not just virtue signalling, and fostering deaf leadership in AI sign language research. We argue for co-creation as a transformative activity that fundamentally alters the status quo and levels the playing field. This necessitates increasing the number of deaf researchers and enhancing AI literacy among deaf communities. Without these critical transformative actions, co-creation risks merely paying lip service to deaf communities.

• The paper illustrates how co-creation in EU sign language tech projects exposes the significant, yet underrecognized, invisible labor of deaf partners.
• It employs iterative dialogue and informal observations, highlighting misaligned expectations and technological readiness challenges.
• The study advocates for inclusive design practices by promoting authentic deaf leadership and diverse participant engagement across projects.

Inclusive Sign Language Technology Development: Truths and Lessons

The paper under analysis provides a comprehensive examination of co-creation within the European Union of the Deaf's (EUD) involvement in two significant EU Horizon 2020 projects: EASIER and SignON. Both projects aimed to develop mobile applications facilitating machine translation between signed and spoken languages. Through a critical exploration grounded in iterative dialogue and informal participant observations, the authors, Maartje De Meulder (HU University of Applied Sciences Utrecht) et al., share both the successes and challenges faced by deaf communities in these consortia.

Summary of Projects

EASIER and SignON represent large-scale collaborations between technologists, predominantly hearing and non-signing, and organizations representing deaf end-users. These projects aimed to create tools for translating between a variety of European signed and spoken languages, leveraging advanced AI technologies such as machine translation, sign language recognition, and 3D avatar synthesis.

• EASIER focused on interactive translation for daily communication and professional use, aiming to achieve various technology readiness levels (TRLs) for different components. The translation framework covered seven signed and six spoken languages.
• SignON sought to develop a more extensive mobile application for automatic translation between several signed and spoken European languages, with use scenarios extending from low-stakes daily interactions to urgent communication needs in the hospitality industry.

Key Findings and Challenges

The paper reveals critical issues within the co-creation processes:

1. Invisible Labor and Accessibility Costs: Deaf partners often bore the brunt of the additional, invisible labor associated with ensuring accessibility, particularly in arranging and managing interpreters for meetings. This labor is not always recognized or compensated adequately.
2. Realism over Hyping Technologies: Managing expectations around nascent technologies was crucial. The projects encountered significant challenges in aligning community expectations with the actual capabilities of the technology being developed.
3. Anonymity and Ethical Concerns: The need to adequately anonymize participants' feedback while respecting the dense networks within deaf communities highlighted the complexity of ethical research practices.
4. Co-Creation Fatigue: Researchers noted a phenomenon akin to "research fatigue" within the communities, driven by the repetitive and overlapping nature of co-creation demands across multiple concurrent projects.
5. Diverse and Intersectional Inclusion: The necessity to engage a diverse range of participants, considering intersectional identities and varying backgrounds within the deaf community, was emphasized as central to the validity of the co-creation process.

Lessons for Future Co-Creation

The authors distilled these reflections into seven key lessons for the future of inclusive sign language technology development:

1. Recognition of Invisible Labor: Acknowledging and compensating the additional work performed by deaf partners is essential.
2. Expectations Management: Balancing realistic technological capabilities with early-stage science communication to manage community expectations effectively.
3. Cripping Co-Creation: Ensuring that the processes are inclusive from inception, considering the unique needs of all participants.
4. Exploring Alternative Methods: Mitigating co-creation fatigue by adopting innovative and varied engagement methodologies, such as art-science methods.
5. Intersectional Feedback: Actively seeking feedback from a diverse and representative group of deaf participants.
6. Avoiding Performative Co-Creation: Ensuring that the co-creation is substantive and not merely a performative exercise in tokenism.
7. Fostering Deaf Leadership: Promoting significant leadership roles for deaf researchers within AI and technology development projects, ensuring a shift from participation to genuine influence and decision-making.

Implications and Future Directions

This investigation into EASIER and SignON projects reveals critical insights not only for future SLMT (Sign Language Machine Translation) projects but also for broader applications of co-creation methodologies in AI-driven technology development. By addressing the identified challenges and integrating the lessons highlighted, the potential for more equitable and effective technology solutions increases. Such a shift is paramount to truly leveraging AI technologies that are cognizant of and responsive to the nuanced needs of diverse user communities.

The authors underscore that co-creation, when executed with genuine intentionality and inclusiveness, holds the promise of being a transformative activity that can reconfigure power dynamics, create more robust and contextually aware technologies, and foster deeper, more impactful engagement between technologists and the communities they aim to serve.