The Model Mastery Lifecycle: A Framework for Designing Human-AI Interaction (2408.12781v1)

Abstract: The utilization of AI in an increasing number of fields is the latest iteration of a long process, where machines and systems have been replacing humans, or changing the roles that they play, in various tasks. Although humans are often resistant to technological innovation, especially in workplaces, there is a general trend towards increasing automation, and more recently, AI. AI is now capable of carrying out, or assisting with, many tasks that used to be regarded as exclusively requiring human expertise. In this paper we consider the case of tasks that could be performed either by human experts or by AI and locate them on a continuum running from exclusively human task performance at one end to AI autonomy on the other, with a variety of forms of human-AI interaction between those extremes. Implementation of AI is constrained by the context of the systems and workflows that it will be embedded within. There is an urgent need for methods to determine how AI should be used in different situations and to develop appropriate methods of human-AI interaction so that humans and AI can work together effectively to perform tasks. In response to the evolving landscape of AI progress and increasing mastery, we introduce an AI Mastery Lifecycle framework and discuss its implications for human-AI interaction. The framework provides guidance on human-AI task allocation and how human-AI interfaces need to adapt to improvements in AI task performance over time. Within the framework we identify a zone of uncertainty where the issues of human-AI task allocation and user interface design are likely to be most challenging.

• The paper introduces model mastery and a framework to design adaptive human-AI interactions as models evolve.
• It proposes a four-stage lifecycle—supervision, interaction, uncertainty, and autonomy—to guide task allocation and interface design.
• The framework emphasizes addressing error types and calibrating trust between human expertise and high-performing AI systems.

The Model Mastery Lifecycle for Human-AI Interaction

This paper introduces the concept of "model mastery" to describe the progression of AI performance in relation to human expertise and presents a framework, the Model Mastery Lifecycle, for designing human-AI interaction (HAII) across different stages of AI development. The framework emphasizes adapting HAII strategies as AI models improve, addressing the challenges of task allocation and user interface design, particularly within a "zone of uncertainty" where the relative expertise of humans and AI is unclear.

Defining Model Mastery

The paper defines model mastery as the point where an AI system's superiority to human experts in a specific task is irrefutable. This concept builds upon decades of research comparing human and algorithmic expertise, highlighting that while models often outperform human experts due to their consistency and lack of cognitive biases, acceptance of model mastery lags behind actual performance [dawes1974linear, meehl1954clinical]. (Figure 1) illustrates how human judgments tend to vary more than model predictions, leading to models outperforming humans when consistency is critical.

Figure 1: A simple (2D) illustrative example of potential model mastery, showing how human judgments show greater variation around a line of estimated best fit

The paper notes that traditional ML evaluation metrics may not fully capture the nuances of human vs. AI performance, especially when different types of errors have varying costs. This necessitates careful consideration of error types and their implications when evaluating model mastery.

Human-AI Interaction Strategies

The paper proposes a framework for HAII based on the relative expertise of humans and AI, as depicted in (Figure 2). Different HAII strategies are recommended based on whether the human user is an expert or a common user, and whether the AI model is error-prone or high-performing. For instance, when expert users interact with high-performing models, the options range from granting the model autonomy to implementing supervisory control. For non-expert users, the paper suggests that supervisory control may be problematic due to the user's limited ability to judge the model's advice.

Figure 2: The design space for HAII, showing four quadrants of model performance versus human expertise. Recommended HAII strategies are shown within each quadrant

The paper emphasizes the dynamic nature of AI expertise, noting that AI model performance for a particular task typically improves over time with training. This progression necessitates adapting HAII strategies to match the evolving capabilities of the AI.

The Model Mastery Lifecycle

To capture the dynamic relationship between human and AI expertise, the paper introduces the Model Mastery Lifecycle, illustrated in (Figure 3). This lifecycle consists of four stages: supervision, interaction, a zone of uncertainty, and autonomy. Each stage requires different types of HAII, reflecting the changing roles of humans and AI as the model approaches mastery. The zone of uncertainty, where the relative expertise of models vs. humans is unclear, presents the most significant challenges for HAII design.

Figure 3: The Model Mastery Lifecycle: Stages in the Growth of Model Mastery showing how overall HAII strategy changes as model mastery increases

The paper discusses the transition to higher levels of automation in automated driving, using it as an example to show how the type of HAII changes as AI/automation progresses towards mastery.

Inertia in Accepting Model Mastery

The paper addresses the issue of resistance to accepting model mastery, drawing parallels with historical reluctance to adopt algorithmic decision-making in clinical settings. It identifies common arguments used to resist model mastery, such as the belief that experts can overrule the model in specific cases or that personal knowledge of patients outweighs general statistical relationships. The authors caution against these arguments, citing evidence that experts may not always be aware of when they are in a better position than the model to make a decision.

Conclusion

The paper concludes by emphasizing the importance of recognizing the current stage of AI systems in the model mastery lifecycle and designing task allocations and HAII accordingly. It advocates for a balanced approach, acknowledging the potential of AI while also recognizing the need for human oversight, ethical considerations, and safeguards against model limitations. The HAII designs should strive to provide an appropriate balance between human oversight and automated control. Calibrating trust appropriately is also essential. Each application of AI will need to formulate its own approach to HAII, taking into account its position in the model mastery lifecycle, as well as specific requirements reflecting the properties of the domain within which the application is situated.