Recourse for reclamation: Chatting with generative language models (2403.14467v2)

Abstract: Researchers and developers increasingly rely on toxicity scoring to moderate generative LLM outputs, in settings such as customer service, information retrieval, and content generation. However, toxicity scoring may render pertinent information inaccessible, rigidify or "value-lock" cultural norms, and prevent language reclamation processes, particularly for marginalized people. In this work, we extend the concept of algorithmic recourse to generative LLMs: we provide users a novel mechanism to achieve their desired prediction by dynamically setting thresholds for toxicity filtering. Users thereby exercise increased agency relative to interactions with the baseline system. A pilot study ($n = 30$) supports the potential of our proposed recourse mechanism, indicating improvements in usability compared to fixed-threshold toxicity-filtering of model outputs. Future work should explore the intersection of toxicity scoring, model controllability, user agency, and language reclamation processes -- particularly with regard to the bias that many communities encounter when interacting with generative LLMs.

Extending Algorithmic Recourse to Conversational AI Interactions

Introduction to the Study

Recent advancements in Generative LLMs (GLMs) have significantly influenced various sectors, including customer service, information retrieval, and content generation. To mitigate potential harm, developers commonly deploy toxicity scores, filtering out content deemed offensive or harmful. This paper introduces a novel approach, incorporating algorithmic recourse within GLM interactions to enhance user agency and refine toxicity threshold mechanisms. Through a pilot paper with 30 participants, the researchers explore the utility of allowing users to dynamically adjust toxicity thresholds, thereby customizing their interaction based on personal or context-specific tolerances.

Problem Context and Proposed Solution

Toxicity scoring, while essential for moderating content, can inadvertently restrict access to relevant information and impede the process of language reclamation for marginalized communities. Recognizing these challenges, the authors propose a dynamic threshold system for toxicity filtering, granting users more control over what content is filtered. This system distinguishes between absolute toxicity thresholds set by platforms and user-defined tolerances. Through a two-step feedback mechanism, users can decide to view content flagged by the system and subsequently determine whether similar content should be filtered in future interactions.

Methodological Approach

The paper utilized a within-participants design, comparing a fixed-thresholding system (control condition) with the proposed dynamic-thresholding system (recourse condition). Participants engaged in conversations on the theme of "identity" with a GLM, encountering both system defaults and the recourse mechanism depending on the condition. Post-interaction, participants rated the system's usability and provided qualitative feedback. Analysis centered on the feasibility of the recourse mechanism, user satisfaction, and the types of themes emerging from user experiences.

Findings and Observations

Participants widely exercised the recourse option when available, indicating a strong preference for a more customizable experience. Quantitative analysis revealed improvements in usability scores in the recourse condition, suggesting that dynamic thresholding could enhance user satisfaction. Qualitatively, participants highlighted the impact of safety responses on their interaction strategies and expressed frustration with the default filtering system's limitations. Notably, the mechanism's perceived complexity suggested further optimization is needed to fulfill user expectations of control.

Implications and Future Directions

The research presents algorithmic recourse as a viable strategy for improving dialogue interfaces, emphasizing the importance of user empowerment in interactions with AI. This paper highlights the need for future investigations into optimal threshold settings and the feasibility of expanding user control over AI interactions. Given the pilot nature of this paper, there is a significant opportunity for further research, particularly in understanding diverse user populations' needs and expectations.

Conclusion

The incorporation of algorithmic recourse into GLM interactions represents a promising avenue toward aligning AI outputs with user expectations and societal values. By enabling users to tailor toxicity thresholds, this approach fosters a more inclusive and responsive interactive experience. Continued exploration and refinement of this mechanism will be crucial in realizing its full potential for supporting user agency and mitigating biases in AI-generated content.