Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
Gemini 2.5 Pro
GPT-5
GPT-4o
DeepSeek R1 via Azure
2000 character limit reached

Coaching Copilot: Blended Form of an LLM-Powered Chatbot and a Human Coach to Effectively Support Self-Reflection for Leadership Growth (2405.15250v1)

Published 24 May 2024 in cs.HC and cs.AI

Abstract: Chatbots' role in fostering self-reflection is now widely recognized, especially in inducing users' behavior change. While the benefits of 24/7 availability, scalability, and consistent responses have been demonstrated in contexts such as healthcare and tutoring to help one form a new habit, their utilization in coaching necessitating deeper introspective dialogue to induce leadership growth remains unexplored. This paper explores the potential of such a chatbot powered by recent LLMs in collaboration with professional coaches in the field of executive coaching. Through a design workshop with them and two weeks of user study involving ten coach-client pairs, we explored the feasibility and nuances of integrating chatbots to complement human coaches. Our findings highlight the benefits of chatbots' ubiquity and reasoning capabilities enabled by LLMs while identifying their limitations and design necessities for effective collaboration between human coaches and chatbots. By doing so, this work contributes to the foundation for augmenting one's self-reflective process with prevalent conversational agents through the human-in-the-loop approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (69)
  1. The Effect of Goal Moderation on the Achievement and Satisfaction of Physical Activity Goals. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 4 (2020), 116:1–116:18. https://doi.org/10.1145/3432209
  2. Guidelines for human-AI interaction. In Proceedings of the 2019 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 3. https://doi.org/10.1145/3290605.3300233
  3. Supporting Experts with a Multimodal Machine-Learning-Based Tool for Human Behavior Analysis of Conversational Videos. CoRR abs/2402.11145 (2024). https://doi.org/10.48550/ARXIV.2402.11145
  4. Riku Arakawa and Hiromu Yakura. 2019. REsCUE: A framework for REal-time feedback on behavioral CUEs using multimodal anomaly detection. In Proceedings of the 2019 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 572. https://doi.org/10.1145/3290605.3300802
  5. Riku Arakawa and Hiromu Yakura. 2020. INWARD: A computer-supported tool for video-reflection improves efficiency and effectiveness in executive coaching. In Proceedings of the 2020 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–13. https://doi.org/10.1145/3313831.3376703
  6. Chris Argyris. 1977. Double loop learning in organizations. Harvard business review 55, 5 (1977), 115–125.
  7. Godfrey T Barrett-Lennard. 1999. Carl Rogers’ helping system: Journey and substance. SAGE Publications, London, UK.
  8. Reviewing reflection: On the use of reflection in interactive system design. In Proceedings of the 2014 ACM Designing Interactive Systems Conference. ACM, New York, NY, 93–102. https://doi.org/10.1145/2598510.2598598
  9. Getting ready for adult healthcare: Designing a chatbot to coach adolescents with special health needs through the transitions of care. Journal of Pediatric Nursing 49 (2019), 85–91. https://doi.org/10.1016/j.pedn.2019.09.004
  10. LeadBoSki: A Smart Personal Assistant for Leadership Support in Video-Meetings. In Companion Publication of the 2021 ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, 19–22. https://doi.org/10.1145/3462204.3481764
  11. Revisiting reflection in HCI: Four design resources for technologies that support reflection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, 1 (2022), 2:1–2:27. https://doi.org/10.1145/3517233
  12. James Bridgeman and Andrea Giraldez-Hayes. 2023. Using artificial intelligence-enhanced video feedback for reflective practice in coach development: Benefits and potential drawbacks. Coaching: An International Journal of Theory, Research and Practice (2023), 1–18. https://doi.org/10.1080/17521882.2023.2228416
  13. Ethical Dilemmas, Mental Health, Artificial Intelligence, and LLM-Based Chatbots. In Proceedings of the 10th International Work-Conference on Bioinformatics and Biomedical Engineering, Vol. 13920. Springer, 313–326. https://doi.org/10.1007/978-3-031-34960-7_22
  14. “Listen to music, listen to yourself”: Design of a conversational agent to support self-awareness while listening to music. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 119:1–119:19. https://doi.org/10.1145/3544548.3581427
  15. Food diary coaching chatbot. In Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Wearable Computers. ACM, New York, NY, 1676–1680. https://doi.org/10.1145/3267305.3274191
  16. Understanding the Coach-Coachee-Client relationship: A conceptual framework for executive coaching. International Coaching Psychology Review 11, 1 (2016), 6–23.
  17. Fred D. Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13, 3 (1989), 319–340. https://doi.org/10.2307/249008
  18. Yogesh K. Dwivedi et al. 2023. Opinion paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management 71 (2023), 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642
  19. The impact of a virtual teaching assistant (chatbot) on students' learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education 19, 1 (2022). https://doi.org/10.1186/s41239-022-00362-6
  20. Gender Bias in Chatbot Design. In Proceedings of the 3rd International Workshop on Chatbot Research and Design. Springer, 79–93. https://doi.org/10.1007/978-3-030-39540-7_6
  21. Daniel C. Feldman and Melenie J. Lankau. 2005. Executive coaching: A review and agenda for future research. Journal of Management 31, 6 (2005), 829–848. https://doi.org/10.1177/0149206305279599
  22. Different chatbots for different purposes: Towards a typology of chatbots to understand interaction design. In Proceedings of the 2019 International Conference on Internet Science. Springer International Publishing, Cham, Switzerland, 145–156. https://doi.org/10.1007/978-3-030-17705-8_13
  23. Corey Ford and Nick Bryan-Kinns. 2023. Towards a reflection in creative experience questionnaire. In Proceedings of the 2023 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 763:1–763:16. https://doi.org/10.1145/3544548.3581077
  24. A chatbot-based coaching intervention for adolescents to promote life skills: Pilot study. JMIR Human Factors 7, 1 (2020), e16762. https://doi.org/10.2196/16762
  25. Anthony M. Grant. 2012. Making positive change: A randomized study comparing solution-focused vs. problem-focused coaching questions. Journal of Systemic Therapies 31, 2 (2012), 21–35. https://doi.org/10.1521/jsyt.2012.31.2.21
  26. Carolin Graßmann and Carsten C Schermuly. 2021. Coaching with artificial intelligence: Concepts and capabilities. Human Resource Development Review 20, 1 (2021), 106–126.
  27. Daniel P. Greenfield and William K. Hengen. 2004. Confidentiality in Coaching. Consulting to Management 15, 1, 9–14.
  28. Peter Hawkins. 2008. The coaching profession: some of the key challenges. Coaching: An International Journal of Theory, Research and Practice 1, 1 (2008), 28–38. https://doi.org/10.1080/17521880701878174
  29. Baek-Kyoo (Brian) Joo. 2005. Executive Coaching: A Conceptual Framework From an Integrative Review of Practice and Research. Human Resource Development Review 4, 4 (2005), 462–488. https://doi.org/10.1177/1534484305280866
  30. Pearl: A technology probe for machine-assisted reflection on personal data. In Proceedings of the 28th ACM International Conference on Intelligent User Interfaces. ACM, New York, NY, 902–918. https://doi.org/10.1145/3581641.3584054
  31. Towards the NESTORE e-Coach: A tangible and embodied conversational agent for older adults. In Proceedings of the 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Wearable Computers. ACM, New York, NY, 1656–1663. https://doi.org/10.1145/3267305.3274188
  32. Richard R. Kilburg. 1997. Coaching and executive character: Core problems and basic approaches. Consulting Psychology Journal: Practice and Research 49, 4 (1997), 281–299. https://doi.org/10.1037/1061-4087.49.4.281
  33. Dyadic Mirror: Everyday Second-person Live-view for Empathetic Reflection upon Parent-child Interaction. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 3 (2020), 86:1–86:29. https://doi.org/10.1145/3411815
  34. Reflection Companion: A Conversational System for Engaging Users in Reflection on Physical Activity. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 2 (2018), 70:1–70:26. https://doi.org/10.1145/3214273
  35. Large Language Models are Zero-Shot Reasoners. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems. Curran Associates, Inc., Red Hook, NY, 22199–22213.
  36. Interacting with educational chatbots: A systematic review. Education and Information Technologies 28, 1 (2023), 973–1018. https://doi.org/10.1007/s10639-022-11177-3
  37. Caring for Vincent: A Chatbot for Self-Compassion. In Proceedings of the 2019 ACM CHI Conference on Human Factors in Computing Systems. ACM, 702:1–702:13. https://doi.org/10.1145/3290605.3300932
  38. Exploring Design Opportunities for Reflective Conversational Agents to Reduce Compulsive Smartphone Use. In Proceedings of the 5th International Conference on Conversational User Interfaces. ACM, 37:1–37:6. https://doi.org/10.1145/3571884.3604305
  39. Incorporating a Reflective Thinking Promoting Mechanism into Artificial Intelligence-Supported English Writing Environments. Interactive Learning Environments 31, 9 (2021), 5614–5632. https://doi.org/10.1080/10494820.2021.2012812
  40. Doug MacKie. 2015. The effects of coachee readiness and core self-evaluations on leadership coaching outcomes: A controlled trial. Coaching: An International Journal of Theory, Research and Practice 8, 2 (2015), 120–136. https://doi.org/10.1080/17521882.2015.1019532
  41. Accompanying reflection processes by an AI-based StudiCoachBot: A study on rapport building in human-machine coaching using self disclosure. In Proceedings of the 23rd HCI International Conference - Late Breaking Papers. Springer, Cham, Switzerland, 439–457. https://doi.org/10.1007/978-3-030-90328-2_29
  42. Examining AI methods for micro-coaching dialogs. In Proceedings of the 2022 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 440:1–440:24. https://doi.org/10.1145/3491102.3501886
  43. Automated vs. human health coaching: Exploring participant and practitioner experiences. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 99:1–99:37. https://doi.org/10.1145/3449173
  44. Frode Moen and Einar Skaalvik. 2009. The effect from executive coaching on performance psychology. International Journal of Evidence Based Coaching & Mentoring 7, 2 (2009), 31–49.
  45. Informing design for reflection. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction. ACM, New York, NY, 1–10. https://doi.org/10.1145/2971485.2971494
  46. Jase Moussa-Inaty. 2015. Reflective Writing through the Use of Guiding Questions. International Journal of Teaching and Learning in Higher Education 27, 1 (2015), 104–113.
  47. Promoting wellbeing with sunny, a chatbot that facilitates positive messages within social groups. In Extended Abstracts of the 2020 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 1–8. https://doi.org/10.1145/3334480.3383062
  48. Training language models to follow instructions with human feedback. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems. Curran Associates, Inc., Red Hook, NY, 27730–27744.
  49. Jonathan Passmore and David Tee. 2023. Can chatbots replace human coaches? Issues and dilemmas for the coaching profession, coaching clients and for organisations. The Coaching Psychologist 19, 1 (2023), 47–54. https://doi.org/10.53841/bpstcp.2023.19.1.47
  50. Exploring the challenges and opportunities in developing systems to improve alcohol use disorder through chatbot technology. In Extended Abstracts of the 2023 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 123:1–123:5. https://doi.org/10.1145/3544549.3585635
  51. Beyond behavior: The coach’s perspective on technology in health coaching. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 670. https://doi.org/10.1145/3290605.3300900
  52. Understanding how eHealth coaches tailor support for weight loss: Towards the design of person-centered coaching systems. In Proceedings of the 2022 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 285:1–285:16. https://doi.org/10.1145/3491102.3501864
  53. Trends, Challenges and Processes in Conversational Agent Design: Exploring Practitioners’ Views through Semi-Structured Interviews. In Proceedings of the 5th International Conference on Conversational User Interfaces. ACM, 13:1–13:10. https://doi.org/10.1145/3571884.3597143
  54. MeetingCoach: An Intelligent Dashboard for Supporting Effective & Inclusive Meetings. In Proceedings of the 2021 ACM CHI Conference on Human Factors in Computing Systems. ACM, 252:1–252:13. https://doi.org/10.1145/3411764.3445615
  55. Anselm L Strauss and Juliet M Corbin. 1990. Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications, Newbury Park, CA.
  56. The potential use of the authenticity scale as an outcome measure in executive coaching. The Coaching Psychologist 7, 1 (2011), 16–25.
  57. Nicky Terblanche and Danie Cilliers. 2020. Factors that influence users’ adoption of being coached by an artificial intelligence coach. Philosophy of Coaching: An International Journal 5, 1 (2020), 61–70. https://doi.org/10.22316/poc/05.1.06
  58. Performance matters: Students’ perceptions of artificial intelligence coach adoption factors. Coaching: An International Journal of Theory, Research and Practice 16, 1 (2022), 100–114. https://doi.org/10.1080/17521882.2022.2094278
  59. User acceptance of information technology: Toward a unified view. MIS Quarterly 27, 3 (2003), 425–478. https://doi.org/10.2307/30036540
  60. SelVReflect: A guided VR experience fostering reflection on personal challenges. In Proceedings of the 2023 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 323:1–323:17. https://doi.org/10.1145/3544548.3580763
  61. Fridolin: Participatory Design and Evaluation of a Nutrition Chatbot for Older aAults. i-com 22, 1 (2023), 33–52. https://doi.org/10.1515/ICOM-2022-0042
  62. Virtual coaches: Background, theories, and future research directions. Business & Information Systems Engineering 64, 4 (2022), 515–528.
  63. Co-active coaching: New skills for coaching people toward success in work and life. Nicholas Brealey, Boston, MA.
  64. Robert Witherspoon and Randall P White. 1996. Executive coaching: A continuum of roles. Consulting Psychology Journal: Practice and Research 48, 2 (1996), 124–133. https://doi.org/10.1037/1061-4087.48.2.124
  65. Supporting autonomous motivation for physical activity with chatbots during the COVID-19 pandemic: Factorial experiment. JMIR Formative Research 7 (2023), e38500. https://doi.org/10.2196/38500
  66. A Script for Conversational Reflection Guidance: A Field Study on Developing Reflection Competence With Apprentices. IEEE Transactions on Learning Technologies 15, 5 (2022), 554–566. https://doi.org/10.1109/TLT.2022.3207226
  67. The authentic personality: A theoretical and empirical conceptualization and the development of the authenticity scale. Journal of Counseling Psychology 55, 3 (2008), 385–399. https://doi.org/10.1037/0022-0167.55.3.385
  68. The ”conversation” about loss: Understanding how chatbot technology was used in supporting people in grief. In Proceedings of the 2023 ACM CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 646:1–646:15. https://doi.org/10.1145/3544548.3581154
  69. Virtual avatar-based life coaching for children with autism spectrum disorder. Computer 53, 2 (2020), 26–34. https://doi.org/10.1109/MC.2019.2915979
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets