Breaking Barriers or Building Dependency? Exploring Team-LLM Collaboration in AI-infused Classroom Debate (2501.09165v2)

Abstract: Classroom debates are a unique form of collaborative learning characterized by fast-paced, high-intensity interactions that foster critical thinking and teamwork. Despite the recognized importance of debates, the role of AI tools, particularly LLM-based systems, in supporting this dynamic learning environment has been under-explored in HCI. This study addresses this opportunity by investigating the integration of LLM-based AI into real-time classroom debates. Over four weeks, 22 students in a Design History course participated in three rounds of debates with support from ChatGPT. The findings reveal how learners prompted the AI to offer insights, collaboratively processed its outputs, and divided labor in team-AI interactions. The study also surfaces key advantages of AI usage, reducing social anxiety, breaking communication barriers, and providing scaffolding for novices, alongside risks, such as information overload and cognitive dependency, which could limit learners' autonomy. We thereby discuss a set of nuanced implications for future HCI exploration.

• The paper presents an empirical study that investigates AI-tool integration in classroom debates and identifies diverse human-AI interaction strategies.
• The study found that students adopted varied roles when collaborating with AI, optimizing debate performance through dynamic role division.
• The research highlights that while AI enhances information access and reduces social anxiety, it also raises risks of information overload and dependency.

Insights into Team-LLM Collaboration in AI-Infused Classroom Debates

The paper "Breaking Barriers or Building Dependency? Exploring Team-LLM Collaboration in AI-infused Classroom Debate" by Zhang et al. conducts an empirical paper investigating the integration of LLM-based AI tools in classroom debates. The research provides a detailed exploration of the dynamics of human-AI collaboration in a fast-paced, collaborative educational setting and highlights both the advantages and potential pitfalls of this integration.

Main Findings

The authors conducted their paper in a Design History course over four weeks, organizing three rounds of debates involving 22 students. This structured investigation focused on team interaction with AI during debates and how AI assists in collaborative learning. The primary findings showcase a diverse array of interaction patterns that emerge when learners collaborate with AI:

1. Diverse Questioning Approaches: Participants employed several strategies for engaging with AI, including providing keywords, detailing background information, and exploring counter-arguments. This indicates that learners adapt their interaction style based on the type of information or support needed from the AI.
2. Utilization of AI-Generated Content: Learners displayed varying behaviors when processing AI output, ranging from direct integration into arguments to selectively filtering and complementing AI suggestions with additional research. This flexibility underscores the role of AI in facilitating deeper thinking rather than simply dictating the discussion.
3. Collaborative Dynamics and Role Division: The paper identifies the emergent division of labor within groups, where roles such as AI user, information gatherer, content evaluator, and ad-hoc tasker were naturally assumed by participants. This role differentiation was instrumental in optimizing the use of AI in a time-constrained debate setting.
4. Advantages of AI in Learning: AI tools were reported to reduce social anxiety, facilitate communication, and provide a scaffold for novice debaters. Participants benefited from AI's ability to generate substantial content rapidly, which encouraged broad perspective exploration and supported both low-order and higher-order cognitive processes.
5. Risks of AI Integration: While providing numerous benefits, AI also poses challenges such as the risk of information overload and cognitive dependency. Participants reported struggling with the volume of AI-generated content, potentially leading to reduced autonomous critical thinking and over-reliance on AI's ready-made answers.

Implications and Future Directions

This research contributes to the ongoing discourse on human-AI collaboration by emphasizing the unique challenges and opportunities in a classroom setting. It underscores the need for carefully designed AI interfaces that can better facilitate real-time, multi-user interactions while managing information presentation to mitigate overload.

The findings suggest several design implications to enhance AI’s educational utility:

• Improved Information Sharing: Future AI designs should focus on intuitive interfaces that support real-time, collaborative interaction, leveraging multimodal technologies to streamline communication and information flow.
• Contextual Information Presentation: AI tools could be enhanced by providing context-specific explanations and breaking down complex queries to help users process information more effectively, thus supporting mutual growth.
• Dynamic Role Adjustment in Teams: Implementing mechanisms that dynamically adjust AI's involvement and the learners' roles based on confidence levels or task requirements could support a balanced division of labor and sustain engagement.

This work paves the way for further exploration into optimizing AI-supported learning environments. Future research might explore extending these findings across diverse cultural contexts and educational frameworks, addressing the limitations posed by sample homogeneity and the nascent maturity of LLM technology in such settings. Additionally, the paper highlights the potential for research into advanced AI systems that can offer more sophisticated, context-aware educational support in high-paced learning environments.