Designing a Dashboard for Transparency and Control of Conversational AI

Abstract: Conversational LLMs function as black box systems, leaving users guessing about why they see the output they do. This lack of transparency is potentially problematic, especially given concerns around bias and truthfulness. To address this issue, we present an end-to-end prototype-connecting interpretability techniques with user experience design-that seeks to make chatbots more transparent. We begin by showing evidence that a prominent open-source LLM has a "user model": examining the internal state of the system, we can extract data related to a user's age, gender, educational level, and socioeconomic status. Next, we describe the design of a dashboard that accompanies the chatbot interface, displaying this user model in real time. The dashboard can also be used to control the user model and the system's behavior. Finally, we discuss a study in which users conversed with the instrumented system. Our results suggest that users appreciate seeing internal states, which helped them expose biased behavior and increased their sense of control. Participants also made valuable suggestions that point to future directions for both design and machine learning research. The project page and video demo of our TalkTuner system are available at https://bit.ly/talktuner-project-page

• The paper presents a novel dashboard called TalkTuner that reveals and controls internal representations of user attributes in LLMs.
• It employs linear probes and synthetic data to map demographic influences like age, gender, education, and socioeconomic status within models such as LLaMa2Chat-13B.
• User studies demonstrated that dashboard transparency boosts engagement and trust while raising concerns over privacy and inherent biases.

Designing a Dashboard for Transparency and Control of Conversational AI

This paper explores a novel approach to enhancing transparency and user control in conversational AI systems through the development of a dashboard interface. The paper addresses the limitations of current LLMs in terms of user transparency, proposing a system that renders the internal user models of conversational agents visible and controllable.

Interpretability and User Models

The authors begin by establishing the premise that LLMs often operate as black boxes, obscuring the internal processes through which they generate responses tailored to user characteristics such as age, gender, and socioeconomic status. To address this opacity, the paper leverages interpretability techniques to identify linear representations of user attributes within the LLM’s internal states.

The research employs linear probes to detect and quantify user attributes, focusing on age, gender, education, and socioeconomic status. These probes reveal that specific residual activations in the LLaMa2Chat-13B model—a LLM optimized for chat—exhibit strong correlations to these attributes, which can be tapped to render the internal model more transparent.

Synthetic Data and Probing Techniques

Due to the unavailability of extensive labeled conversation datasets, the authors utilize synthetic data generated via role-played interactions with LLMs like GPT-3.5 and LLaMa2Chat. This synthetic dataset aids in training logistic regression probes to interpret the internal representation of user attributes accurately.

The paper details an innovative causal intervention experiment, which modifies user attribute representations within the model to observe resultant changes in behavior and response accuracy. Control probes are introduced, which alter these attributes by translating the learned internal representations.

Figure 1: Effect of training data size on the reading and control probe's performance. The accuracy is measured on a held-out validation set of each attribute.

Dashboard Design and User Interaction

The core of the paper is the description of a dashboard named TalkTuner, which is designed to make these internal user models available to end users. The dashboard provides real-time visualization and manual control over user attributes, allowing users to 'pin' demographic attributes that affect conversational output.

The design methodology combines interpretability with user-experience design to ensure users not only see the model’s internal state but can meaningfully interact with and adjust it.

Figure 2: Dashboard interface. (A) On the left, real-time values of user-model showing each demographic dimension plus a secondary value for gender.

User Study Findings

Through a user study involving diverse participants, the researchers confirmed that transparency through the dashboard enhanced user engagement and trust in the AI system. The study revealed that participants appreciated seeing the decision-making process of the chatbot, although biases detected through the dashboard sometimes reduced trust in the system. Additionally, users expressed privacy concerns and were occasionally discomforted by the model’s assumptions.

The study also indicated that allowing users to control the chatbot’s internal model corrected incorrect assumptions and personalized interactions effectively. Users valued the ability to experiment with alternate demographic settings to uncover biases.

Figure 3: Questionnaire responses with Wilcoxon signed rank test. See Appendix~\ref{appendix:post-task-questionnaires}.

Implications and Future Research

This research has significant implications for both AI interpretability and user control. It points towards potential future enhancements in chatbot interfaces that include diverse user models, more intricate and comprehensive user feedback mechanisms, and broader conversational attribute adjustments. The authors suggest further exploration into security and privacy concerns, as well as the generalization of the dashboard concept to other AI applications such as voice and video bots.

The deployment of interactive interpretability tools represents a tangible advancement in human-AI interaction, offering users unprecedented insight and authority over algorithmic processes. This represents a crucial step towards more user-friendly, transparent, and trustworthy AI systems.

Conclusion

The paper presents a comprehensive approach for bridging the gap between complex LLM systems and user interaction through innovative visualization and control interfaces. By combining interpretability techniques with user-centered design, this research advances towards a future where conversational AI systems are transparent, controllable, and engender greater trust among users. Future explorations are warranted to refine these interfaces and address privacy considerations, ensuring that this model of transparency can be applied across a broader spectrum of AI applications.