A Map of Exploring Human Interaction patterns with LLM: Insights into Collaboration and Creativity (2404.04570v1)

Abstract: The outstanding performance capabilities of LLM have driven the evolution of current AI system interaction patterns. This has led to considerable discussion within the Human-AI Interaction (HAII) community. Numerous studies explore this interaction from technical, design, and empirical perspectives. However, the majority of current literature reviews concentrate on interactions across the wider spectrum of AI, with limited attention given to the specific realm of interaction with LLM. We searched for articles on human interaction with LLM, selecting 110 relevant publications meeting consensus definition of Human-AI interaction. Subsequently, we developed a comprehensive Mapping Procedure, structured in five distinct stages, to systematically analyze and categorize the collected publications. Applying this methodical approach, we meticulously mapped the chosen studies, culminating in a detailed and insightful representation of the research landscape. Overall, our review presents an novel approach, introducing a distinctive mapping method, specifically tailored to evaluate human-LLM interaction patterns. We conducted a comprehensive analysis of the current research in related fields, employing clustering techniques for categorization, which enabled us to clearly delineate the status and challenges prevalent in each identified area.

• The paper proposes a five-stage mapping methodology that categorizes human-LLM interactions into four clusters: Processing Tool, Analysis Assistant, Creative Companion, and Processing Agent.
• The study reveals that the Creative Companion cluster dominates current research, indicating LLMs’ growing role in augmenting human creativity and facilitating autonomous decision-making.
• The research identifies gaps in advisory insights and transparency, suggesting the need to improve AI explainability in collaborative decision-making processes.

Insights into Human Interaction Patterns with LLMs

Introduction

The paper "A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and Creativity" (2404.04570) presents a comprehensive investigation into the interaction dynamics between humans and LLMs. This paper fills a notable gap in the Human-AI Interaction (HAII) literature by focusing specifically on LLMs, which have transformed AI systems' capabilities, particularly in creative tasks, general intelligence, and language understanding. The mapping procedure introduced here aims to systematically categorize existing research on these interactions, facilitating a clearer understanding of where current efforts are concentrated and what challenges remain.

Mapping Methodology

The authors adopted a five-stage mapping methodology to analyze relevant studies on human-LLM interactions. The procedure involves defining research perspectives, identifying dimensions, associating relevant concepts, establishing evaluation criteria, and scoring and clustering publications.

Figure 1: Flow diagram of the database searches and article screenings.

Collaboration and Creativity: The paper frames its mapping on two dimensions: Human-AI interaction, which evaluates the collaborative relationships and decision-making dynamics between humans and LLMs, and Implement-Creation, which reflects the creative contributions of AI in various tasks.

Detailed Evaluation: Through rigorous literature screening, the authors selected 110 papers from an initial pool of 1398, focusing on those that explicitly engage with Human-LLM interaction patterns. Criteria involved whether papers presented user studies centered on direct interactions with LLM systems, rather than solely domain-specific applications.

Clustering and Mapping Results

The paper categorizes Human-LLM interaction into four distinct clusters: Processing Tool, Analysis Assistant, Creative Companion, and Processing Agent.

Figure 2: Types of task division between human and AI in work.

Figure 3: Innovation level of content produced through human-AI task.

Processing Tool: AI functions as a tool performing specific data processing tasks with minimal creative input, facilitating decision-making predominantly dominated by humans.

Analysis Assistant: AI aids in analytical tasks, generating insights and supporting human decisions with a creative flair in data interpretation.

Creative Companion: In this role, LLMs show significant autonomy, contributing creatively to tasks and collaborating closely with human users.

Processing Agent: AI independently handles complex tasks that involve organizing and summarizing data, making autonomous decisions similar to the role of Creative Companions.

Current Research Trends and Implications

The paper highlights a strong research focus on the Creative Companion cluster, indicating a shift towards leveraging LLMs' generative capabilities. These models can extend human ideas creatively, facilitating more autonomous decision-making in collaborative settings.

Figure 4: Map of the field of human-LLM interaction patterns. The chart displays the comparative volume of research papers in each sector.

Emerging Challenges: The vacancy in the map suggests limited exploration of AI's role in providing advisory insights with high transparency, which requires both creative output and contribution to collaborative decision-making. Future research may need to address these gaps by enhancing AI's explanatory capabilities.

Future Directions

The evolution of LLM understanding and generative abilities will likely expand their role across various collaborative applications. Future studies should focus on diversifying task allocations, refining AI autonomy levels, and exploring innovative human-AI collaboration frameworks.

Conclusion

This paper provides a structured approach to evaluating and understanding Human-LLM interaction patterns, emphasizing the growing autonomy and creativity of LLMs in collaborative environments. The insights garnered from this research will aid in pinpointing emerging areas, evaluating current systems, and identifying future challenges within the field.