- The paper identifies gaps in traditional dyadic CPS studies by emphasizing the need for scalable environments that simulate complex multiagent and human-AI interactions.
- The paper introduces CPS-TaskForge, a tower defense-based generator that flexibly adjusts task parameters like team composition and resource management to study CPS dynamics.
- The paper reports that participant interactions featured 49% cognitive skills and 29% planning activities, validating the tool’s effectiveness in eliciting rich collaborative problem-solving behaviors.
CPS-TaskForge: Generating Collaborative Problem Solving Environments for Diverse Communication Tasks
The paper "CPS-TaskForge: Generating Collaborative Problem Solving Environments for Diverse Communication Tasks" introduces a novel tool for generating environments conducive to the study of Collaborative Problem Solving (CPS) across various conditions—CPS-TaskForge. This tool aims to address the existing data challenges in CPS research, particularly the paucity of open data involving more than dyadic interactions.
CPS-TaskForge is predicated on the notion that CPS tasks with more complex team compositions can yield richer insights, especially with the integration of AI teammates. The primary innovation is a CPS task generator that produces environments using a resource management game paradigm (specifically, a tower defense game). This platform can manipulate different CPS task design parameters, therefore allowing researchers to study CPS in diverse settings. Furthermore, the authors provide a CPS task design checklist grounded in the PISA 2015 CPS framework which facilitates the systematic development of CPS environments.
Key Contributions:
- Identification of Research Gaps:
- The paper underscores the necessity of research on teams with more than two agents. It argues that existing empirical CPS studies largely focus on dyadic interactions, thereby neglecting the diversity and complexity of larger team dynamics.
- Furthermore, it emphasizes that datasets involving human-AI teams are limited, primarily due to the lack of adaptable, scalable task generators.
- Introduction of CPS-TaskForge:
- CPS-TaskForge is capable of generating a variety of CPS task environments. The primary task environment is a tower defense game which involves planning and resource management—both critical aspects of CPS.
- The tool allows for the adjustment of various CPS design parameters such as team composition, communication modality, task difficulty, resource allocation, and more, thereby enabling detailed and nuanced studies of CPS processes.
- Case Study Validation:
- The authors conducted a case study with groups of 3-4 humans, demonstrating the tool’s ability to generate diverse and natural language CPS communication.
- The study explored various CPS skills, including social skills (maintaining communication, sharing information) and cognitive skills (planning, executing actions).
Numerical Results and Observations:
- Diverse Strategies and CPS Skills:
- Participants employed different strategies for resource management, showcasing the task environment's flexibility and its capacity to elicit varied CPS behaviors. One team might focus on concentrated defenses, while another might distribute resources across the map.
- Communication was rich, with cognitive skills employed 49% of the time and planning activities comprising 29% of team interactions, supporting the tool’s efficacy in generating meaningful CPS data.
- Positive Participant Reception:
- Participants noted the task's engaging nature, with many expressing interest in playing outside of the study context, validating the design requirement for the task to be entertaining and motivation-inducing.
Implications and Future Directions:
The development of CPS-TaskForge has significant theoretical and practical implications. Theoretically, the tool enhances our understanding of team dynamics and collaborative problem-solving processes in multi-agent settings, including human-AI teams. Practically, it provides a scalable and adaptable platform for researchers to systematically explore various facets of CPS.
Future Developments:
- Incorporation of more sophisticated game mechanics to simulate even more diverse CPS scenarios.
- Extension to support different communication modalities and longitudinal studies to observe CPS processes over time.
- Exploration of human-AI interaction paradigms, particularly focusing on trust, reliance, and efficacy in hybrid teams.
Concluding Remarks:
CPS-TaskForge represents a substantial step forward in CPS research, offering a versatile and theoretically grounded platform for generating CPS task environments. This tool opens new avenues for studying collaborative efforts in more complex, larger team settings and promotes a deeper integration of AI in team problem-solving activities.