- The paper compares facilitator and peer generative AI agents in six four-person groups solving digital escape-room puzzles, finding that the facilitator condition achieved the highest performance score (14.50) while the peer condition scored lowest (4.67).
- The peer agent significantly increased workload and disrupted group flow, despite receiving higher ratings for improving coordination, revealing a strong gap between perceived helpfulness and objective team performance.
- The findings suggest that collaborative AI should deliver concise, activity-aware support with clear rationales, adjustable initiative, and timing based on team stalls rather than fixed intervention intervals.
This paper reports a within-subjects mixed-methods study examining how two distinct proactive generative AI agent roles—a process-oriented facilitator and a task-oriented peer—influence group performance and processes in co-located, time-sensitive collaborative problem-solving (2602.17864). Conducted with six four-person groups (24 participants total) solving digital escape-room puzzles under a 20-minute time limit, the study compares three conditions: no AI, a peer agent ("Ava"), and a facilitator agent ("Fiona"). The central finding is counterintuitive and worth emphasizing: the peer agent, which participants perceived as more helpful, was associated with the worst objective performance, while the largely ignored facilitator condition produced the highest scores.
Study context and motivation
The work sits at the intersection of three research threads: generative AI as a collaborator, proactive systems that act without explicit prompting, and the role-based framing of human-AI teams. The authors note that most prior evidence on proactive AI in collaboration derives from speculative design or wizard-of-oz studies; their contribution is to deploy functional technology probes in a demanding, ecologically valid setting. Escape rooms were chosen deliberately: they require tightly coupled coordination across distributed visual clues, and—critically—the puzzles were designed so that state-of-the-art multimodal models (GPT-4.1, o3, o4) could not solve them alone when viewed in isolation. This ensured the agents had to function as imperfect collaborators constructing partial answers alongside humans rather than as oracles.
Agent design
Both probes were grounded in prior literature and refined through a formative study (six participants) and pilot (four participants). Fiona, the facilitator, scaffolded metacognition through collaboration strategy prompts (e.g., the 1-2-4-All liberating structure), time reminders, a task-division prompt at the 13-minute mark, and periodic discussion summaries generated every three minutes from real-time WhisperX transcription, grounded in puzzle screenshots via GPT-4.1-mini. Ava, the peer, delivered six pre-generated proactive "thoughts" at three-minute intervals (generated by o3, contextualized at runtime by gpt-4o-mini), offered interactive follow-up buttons, and served as an embedded chat partner powered by gpt-4.1-mini. Notably, only roughly half of Ava's thoughts connected the correct elements across screens, making her an intentionally imperfect teammate whose limitations were disclosed to participants.
Quantitative results
The analysis used ART-ANOVA with post-hoc ART-c contrasts, appropriate given bounded Likert data and small group-level samples. Three results stand out:
- Group performance favored the facilitator despite its perceived redundancy. The main effect of AI Condition was significant (F=13.33, p=.006), with estimated marginal means of 14.50 (facilitator), 9.33 (no AI), and 4.67 (peer). The facilitator outperformed the peer with a very large effect size (d=−2.98). Puzzle difficulty was also significant, with Puzzle 1 substantially easier than Puzzles 2 and 3.
- The peer agent significantly increased workload. NASA-TLX scores were highest in the peer condition (EMM ≈ 47.5 versus ≈ 30–32 for the other conditions), differing significantly from both the facilitator (d=−0.88) and no-AI (d=−0.82) conditions.
- Perceived benefit diverged from measured outcomes. Participants rated the peer significantly higher than the facilitator on "improved our team coordination" (d=6.39), yet no significant differences emerged for trustworthiness, interpretability, comfort depending on the agent, or perceived impact on team score.
This perception–performance dissociation is the paper's most consequential claim. The authors argue it arises from visibility asymmetries: Ava's interventions were immediate, salient, and attributable, whereas Fiona's summaries blended into ongoing conversation and supported coordination indirectly, so teams never credited her for their higher scores.
Qualitative findings
Reflexive thematic analysis surfaced role-specific dynamics. The facilitator provided early structural guidance that persisted even after its suggestions ceased, and occasionally anchored shared focus productively; however, all six groups judged its overall performance impact minimal, describing summaries as redundant under time pressure. Twenty of twenty-four participants reported forgetting or ignoring the agent as sessions progressed.
The peer agent showed sharper contrasts. Its timely hints and memory/calculation offloading demonstrably helped some groups—one participant stated that "most of the tasks we solved were hints given by the agent." Yet fourteen of twenty-four participants reported flow disruption, ten reported added cognitive burden from prompt composition and verbose output, and eight described trust erosion following confident outputs delivered without reasoning. Chat engagement was skewed: sixteen of twenty-four participants interacted zero to three times, while five showed high engagement, and the dedicated follow-up buttons were used only twice across all sessions.
The paper's third peer theme documents heterogeneous adoption trajectories—reliance maturing into reflection (Groups 2–3), enthusiasm collapsing into dependence and disillusionment (Groups 1–4), and curiosity degrading into frustration and disengagement (Groups 5–6). This heterogeneity cautions against uniform conclusions about proactive AI value and suggests intervention timing and group state jointly determine outcomes.
Design considerations
From these findings the authors derive concrete design guidance: contributions should be concise, progress-aware, and embedded in the shared workspace; suggestions should carry rationales and, where feasible, multiple alternatives to mitigate anchoring; agents should maintain moderate social presence; user controls over initiative (rate limits, silence thresholds, pause options) should be exposed; and timing should be driven by activity-based triggers such as stalls in conversation rather than fixed intervals. The authors are appropriately careful here, noting that their findings reflect the interaction of role and specific feature implementations, so implications are framed at the feature level rather than as claims about abstract roles.
Limitations
Several constraints bound the generalizability of these results. Participants were colleagues who knew one another, which likely inflated baseline coordination ratings (which did not differ across conditions) and shaped communication patterns. The escape-room testbed involves short-horizon, self-contained sub-tasks that may not transfer to domains like healthcare or crisis response. Each agent performed a single non-adaptive role with fixed-interval interventions; neither adapted to group progress or experience. Finally, with six groups, statistical power is limited, and several pairwise comparisons (e.g., facilitator vs. no AI on performance, p=.070) approached but did not reach significance.
Conclusion
This study provides empirical evidence that the visibility of an AI agent's contributions can be decoupled from—and even inversely related to—their effect on team outcomes in time-pressured collaboration. A quiet facilitator measurably improved performance while being perceived as useless; a salient peer raised workload, fostered over-reliance, fragmented communication into silos, and was nonetheless credited as more helpful. The open question the paper leaves is how to build adaptive agents that modulate between facilitative and peer-like support based on real-time signals of group state, timing interventions at moments of genuine need rather than on fixed schedules.