The System Model and the User Model: Exploring AI Dashboard Design (2305.02469v1)

Abstract: This is a speculative essay on interface design and artificial intelligence. Recently there has been a surge of attention to chatbots based on LLMs, including widely reported unsavory interactions. We contend that part of the problem is that text is not all you need: sophisticated AI systems should have dashboards, just like all other complicated devices. Assuming the hypothesis that AI systems based on neural networks will contain interpretable models of aspects of the world around them, we discuss what data such dashboards might display. We conjecture that, for many systems, the two most important models will be of the user and of the system itself. We call these the System Model and User Model. We argue that, for usability and safety, interfaces to dialogue-based AI systems should have a parallel display based on the state of the System Model and the User Model. Finding ways to identify, interpret, and display these two models should be a core part of interface research for AI.

• The paper demonstrates that AI dashboards revealing internal models substantially enhance transparency and trust.
• It employs design experiments and real-world examples, like Tesla displays, to validate the dashboard utility hypothesis.
• Key insights emphasize that displaying both user and system models can improve safety and reliability in human-AI interactions.

Exploring AI Dashboard Design: Insights into the System Model and User Model

The paper "The System Model and the User Model: Exploring AI Dashboard Design," authored by Fernanda Viégas and Martin Wattenberg, addresses a pressing issue in the field of human-AI interaction: the need for sophisticated AI systems to have dashboards that provide real-time information about their internal state. The authors argue that effective human-AI interaction transcends mere conversation, necessitating accessible and interpretable AI system dashboards.

Core Hypothesis and Conjectures

The paper is founded on two conjectures:

1. Interpretable Model Hypothesis: Neural networks inherently contain interpretable models of the world they interact with.
2. Dashboard Utility Hypothesis: Visualization and display of simplified data from these models are immensely beneficial to users.

The authors posit that AI systems, akin to mechanical devices, benefit from instrumentation that conveys their internal state. For instance, the Tesla touchscreen, which displays an inferred state of the road ahead, aids drivers in calibrating their trust in the system. The idea here is that even when world models are not explicitly built into AI systems, they exist and can be surfaced for user interfaces.

User Model and System Model

Viégas and Wattenberg emphasize two essential models within AI systems that should be prominently accessible:

• User Model: This model represents the AI's interpretation of the user, including attributes such as gender, age, and location. For instance, ChatGPT’s adjustment from masculine to feminine forms of address based on conversational cues implies a model of the user's gender. Displaying such a model would provide users with transparency about how the system interprets and reacts to their inputs.
• System Model: This model reflects the AI's understanding of its own state and behavior. For instance, distinguishing whether a LLM is generating fiction or non-fiction could significantly aid users in calibrating trust and understanding the AI’s outputs.

Implications and Design Considerations

The practical implications of these insights are vast. For AI systems to be safe and reliable, especially in high-stakes applications such as medical advice, users must understand the basis of the AI’s inferences and actions. The authors advocate for interfaces that clearly display the AI’s models of the user and the AI itself, thereby mitigating potential risks and enhancing usability.

Choosing Display Features

Determining which features of the User and System Models to display requires extensive experimentation. Important considerations include:

• Relevance and Usefulness: Features must provide meaningful insights without overwhelming the user.
• Safety and Trust Calibration: Features that affect safety and trust, such as indicators of the AI’s confidence or state, should be prominently displayed.
• User Preferences: Some users may find certain features, like an inferred skill level, reassuring, while others may find them offensive.

Future Research Directions

The paper calls for a targeted research program focusing on identifying, interpreting, and displaying AI models. Key areas for further investigation include:

• Model Identification and Extraction Techniques: Developing robust methods for uncovering and interpreting world models within neural networks.
• Interface Design Best Practices: Experimenting with various display approaches to determine the most effective ways to present internal state information.
• Safety and Ethical Considerations: Ensuring that displayed features enhance user safety and trust without introducing new risks or ethical concerns.

Conclusion

"The System Model and the User Model: Exploring AI Dashboard Design" sheds light on a critical and often overlooked aspect of human-AI interaction. Viégas and Wattenberg’s arguments underscore the necessity for AI systems to have clear and interpretable dashboards that display their internal states, specifically the User and System Models. This approach holds the promise of making AI interactions safer, more transparent, and more reliable. Future research in this area is essential to fully realize these benefits and develop best practices for AI dashboard design.