Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding Human-AI Collaboration in Music Therapy Through Co-Design with Therapists (2402.14503v3)

Published 22 Feb 2024 in cs.HC

Abstract: The rapid development of musical AI technologies has expanded the creative potential of various musical activities, ranging from music style transformation to music generation. However, little research has investigated how musical AIs can support music therapists, who urgently need new technology support. This study used a mixed method, including semi-structured interviews and a participatory design approach. By collaborating with music therapists, we explored design opportunities for musical AIs in music therapy. We presented the co-design outcomes involving the integration of musical AIs into a music therapy process, which was developed from a theoretical framework rooted in emotion-focused therapy. After that, we concluded the benefits and concerns surrounding music AIs from the perspective of music therapists. Based on our findings, we discussed the opportunities and design implications for applying musical AIs to music therapy. Our work offers valuable insights for developing human-AI collaborative music systems in therapy involving complex procedures and specific requirements.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (99)
  1. Brian Abrams. 2018. Understanding humanistic dimensions of music therapy: Editorial introduction. , 139–143 pages.
  2. Steven J Ackerman and Mark J Hilsenroth. 2003. A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clinical psychology review 23, 1 (2003), 1–33.
  3. Kenneth Aigen. 2014. Music-centered dimensions of Nordoff-Robbins music therapy. Music Therapy Perspectives 32, 1 (2014), 18–29.
  4. Understanding the Technological Practices and Needs of Music Therapists. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–25.
  5. Lynne M Baker. 2010. Music therapy: Diversity, challenge and impact. International Journal of Disability, Development and Education 57, 3 (2010), 335–340.
  6. Exploring the creation of useful interfaces for music therapists. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion. 1–7.
  7. Lora Humphrey Beebe and Tami H Wyatt. 2009. Guided imagery & music: Using the bonny method to evoke emotion & access the unconscious. Journal of psychosocial nursing and mental health services 47, 1 (2009), 29–33.
  8. Ronaldo O Benenzon. 2007. The benenzon model. Nordic Journal of Music Therapy 16, 2 (2007), 148–159.
  9. MoMusic: a motion-driven human-AI collaborative music composition and performing system. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 16057–16062.
  10. Sumitra Biswal. 2021. I Can Feel You: A Self-Trained Therapeutic Music Recommender System. In 2021 11th International Conference on Advanced Computer Information Technologies (ACIT). IEEE, 753–756.
  11. MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer. In Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018, Emilia Gómez, Xiao Hu, Eric Humphrey, and Emmanouil Benetos (Eds.). 747–754. http://ismir2018.ircam.fr/doc/pdfs/204_Paper.pdf
  12. Kenneth E Bruscia. 1988. A survey of treatment procedures in improvisational music therapy. Psychology of Music 16, 1 (1988), 10–24.
  13. Exploring XAI for the Arts: Explaining Latent Space in Generative Music. CoRR abs/2308.05496. https://doi.org/10.48550/ARXIV.2308.05496 arXiv:2308.05496
  14. Leslie Bunt. 2012. Music therapy: A resource for creativity, health and well-being across the lifespan. Musical creativity: Insights from music education research (2012), 165–181.
  15. Human-centered tools for coping with imperfect algorithms during medical decision-making. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–14.
  16. “Listen to Music, Listen to Yourself”: Design of a Conversational Agent to Support Self-Awareness While Listening to Music. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19.
  17. A new era in Internet interventions: The advent of Chat-GPT and AI-assisted therapist guidance. Internet Interventions 32 (2023).
  18. Shan Carter and Michael Nielsen. 2017. Using artificial intelligence to augment human intelligence. Distill 2, 12 (2017), e9.
  19. Caroline Case and Tessa Dalley. 2014. The handbook of art therapy. Routledge.
  20. Music therapy. (2013).
  21. Investigating positive and negative qualities of human-in-the-loop optimization for designing interaction techniques. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–14.
  22. Effects of music therapy on psychological health of women during pregnancy. Journal of clinical nursing 17, 19 (2008), 2580–2587.
  23. Use of preferred music to reduce emotional distress and symptom activity during radiation therapy. Journal of music therapy 43, 3 (2006), 247–265.
  24. Computer assisted music therapy: A case study of an augmented reality musical system for children with cerebral palsy rehabilitation. In 2009 Ninth IEEE International Conference on Advanced Learning Technologies. IEEE, 218–220.
  25. Associations between therapists’ occupational burnout and their patients’ depression and anxiety treatment outcomes. Depression and anxiety 35, 9 (2018), 844–850.
  26. DeepThInk: Designing and probing human-AI co-creation in digital art therapy. International Journal of Human-Computer Studies 181 (2024), 103139.
  27. DDSP: Differentiable Digital Signal Processing. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/forum?id=B1x1ma4tDr
  28. Johannes Th Eschen. 2002. Analytical music therapy. Jessica Kingsley Publishers.
  29. Music creation by example. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.
  30. Elżbieta Galińska. 2015. Music therapy in neurological rehabilitation settings. Psychiatria polska 49, 4 (2015), 835–846.
  31. Laura H Goetz and Nicholas J Schork. 2018. Personalized medicine: motivation, challenges, and progress. Fertility and sterility 109, 6 (2018), 952–963.
  32. Lori F Gooding. 2019. Burnout among music therapists: An integrative review. Nordic Journal of Music Therapy 28, 5 (2019), 426–440.
  33. Leslie S Greenberg. 2004. Emotion–focused therapy. Clinical Psychology & Psychotherapy: An International Journal of Theory & Practice 11, 1 (2004), 3–16.
  34. Jacek Grekow and Teodora Dimitrova-Grekow. 2021. Monophonic music generation with a given emotion using conditional variational autoencoder. IEEE Access 9 (2021), 129088–129101.
  35. Effect of music therapy on anxiety and depression in patients with Alzheimer’s type dementia: randomised, controlled study. Dementia and geriatric cognitive disorders 28, 1 (2009), 36–46.
  36. SenseBox: A DIY prototyping platform to create audio interfaces for therapy. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction. 25–34.
  37. SophiaPop: Experiments in Human-AI Collaboration on Popular Music. CoRR abs/2011.10363 (2020). arXiv:2011.10363 https://arxiv.org/abs/2011.10363
  38. Carlos Hernandez-Olivan and José Ramón Beltrán. 2021. Music Composition with Deep Learning: A Review. CoRR abs/2108.12290. arXiv:2108.12290 https://arxiv.org/abs/2108.12290
  39. A functional taxonomy of music generation systems. ACM Computing Surveys (CSUR) 50, 5 (2017), 1–30.
  40. Clara E Hill. 2005. Therapist techniques, client involvement, and the therapeutic relationship: Inextricably intertwined in the therapy process. Psychotherapy: theory, research, practice, training 42, 4 (2005), 431.
  41. Yurui Hou. 2022. AI Music Therapist: A Study on Generating Specific Therapeutic Music based on Deep Generative Adversarial Network Approach. In 2022 IEEE 2nd International Conference on Electronic Technology, Communication and Information (ICETCI). IEEE, 1277–1281.
  42. Compound word transformer: Learning to compose full-song music over dynamic directed hypergraphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 178–186.
  43. Make your favorite music curative: Music style transfer for anxiety reduction. In Proceedings of the 28th ACM International Conference on Multimedia. 1189–1197.
  44. Counterpoint by Convolution. In Proceedings of the 18th International Society for Music Information Retrieval Conference, ISMIR 2017, Suzhou, China, October 23-27, 2017, Sally Jo Cunningham, Zhiyao Duan, Xiao Hu, and Douglas Turnbull (Eds.). 211–218. https://ismir2017.smcnus.org/wp-content/uploads/2017/10/187_Paper.pdf
  45. EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation. In International Society for Music Information Retrieval Conference. https://api.semanticscholar.org/CorpusID:236881050
  46. The effects of music therapy on anxiety and depression of cancer patients. Indian journal of palliative care 22, 4 (2016), 455.
  47. FashionQ: an ai-driven creativity support tool for facilitating ideation in fashion design. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.
  48. Understanding Disclosure and Support for Youth Mental Health in Social Music Communities. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1–32.
  49. Effects of personal characteristics in control-oriented user interfaces for music recommender systems. User Modeling and User-Adapted Interaction 30 (2020), 199–249.
  50. William A Kahn. 1990. Psychological conditions of personal engagement and disengagement at work. Academy of management journal 33, 4 (1990), 692–724.
  51. Colorbo: Envisioned Mandala Coloringthrough Human-AI Collaboration. In 27th International Conference on Intelligent User Interfaces. 15–26.
  52. Motivating stroke rehabilitation through music: A feasibility study using digital musical instruments in the home. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1781–1785.
  53. Coping with stress: the effectiveness of different types of music. Applied psychophysiology and biofeedback 32, 3 (2007), 163–168.
  54. Jungup Lee and Bruce A Thyer. 2013. Does music therapy improve mental health in adults? A review. Journal of Human Behavior in the Social Environment 23, 5 (2013), 591–603.
  55. Co-design and evaluation of an intelligent decision support system for stroke rehabilitation assessment. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–27.
  56. A human-ai collaborative approach for clinical decision making on rehabilitation assessment. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–14.
  57. Do therapists’ subjective variables impact on psychodynamic psychotherapy outcomes? A systematic literature review. Clinical psychology & psychotherapy 25, 1 (2018), 85–101.
  58. Chimelight: Augmenting instruments in interactive music therapy for children with neurodevelopmental disorders. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility. 124–135.
  59. Novice-AI music co-creation via AI-steering tools for deep generative models. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–13.
  60. Music, health, and wellbeing. Oxford University Press.
  61. Raymond AR MacDonald and Graeme B Wilson. 2014. Musical improvisation and health: a review. Psychology of Well-being 4, 1 (2014), 1–18.
  62. Tracey McConnell and Sam Porter. 2017. Music therapy for palliative care: a realist review. Palliative & supportive care 15, 4 (2017), 454–464.
  63. Tencent Meeting. 2022. https://www.tencent.com/en-us/responsibility/combat-covid-19-tencent-meeting.html
  64. Controlling spotify recommendations: effects of personal characteristics on music recommender user interfaces. In Proceedings of the 26th Conference on user modeling, adaptation and personalization. 101–109.
  65. Measuring alliance toward embodied virtual therapists in the era of automated treatments with the Virtual Therapist Alliance Scale (VTAS): Development and psychometric evaluation. Journal of medical Internet research 22, 3 (2020), e16660.
  66. Gail C Mornhinweg. 1992. Effects of music preference and selection on stress reduction. Journal of Holistic Nursing 10, 2 (1992), 101–109.
  67. Collaborative artificial intelligence in music production. In Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion. 1–4.
  68. Jason Noone. 2008. Developing a music therapy programme within a person centred planning framework. In Voices: A world forum for music therapy, Vol. 8.
  69. Creative Music Therapy: A Guide to Fostering Clinical Musicianship. Barcelona Pub.
  70. Nicholas Novelli and Shannon Proksch. 2022. Am I (Deep) Blue? Music-Making AI and Emotional Awareness. Frontiers in Neurorobotics 16 (2022), 897110.
  71. I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
  72. OpenAI. 2019. MuseNet. https://openai.com/blog/musenet/
  73. Mercedes Pavlicevic. 1997. Music therapy in context: Music, meaning and relationship. Jessica Kingsley Publishers.
  74. Immersive Music Therapy for Elderly Patients. In ACM International Conference on Interactive Media Experiences. 47–52.
  75. Kevin Puhr. 2018. Current Perceptions of Gender Bias in Music Therapy. (2018).
  76. Generated Therapeutic Music Based on the ISO Principle. In Summit on Music Intelligence. Springer, 32–45.
  77. Grazia Ragone. 2020. Designing embodied musical interaction for children with autism. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1–4.
  78. Towards effective music therapy for mental health care using machine learning tools: human affective reasoning and music genres. Journal of Artificial Intelligence and Soft Computing Research 11 (2021).
  79. A hierarchical latent vector model for learning long-term structure in music. In International conference on machine learning. PMLR, 4364–4373.
  80. Research rigour and therapeutic flexibility: Rationale for a therapy manual developed for a randomised controlled trial. Nordic Journal of Music Therapy 14, 1 (2005), 15–32.
  81. Jessica Rushing and Gilson J Capilouto. 2017. Obtaining competencies with self-determination theory: The music therapy internship. Music Therapy Perspectives 35, 2 (2017), 247–253.
  82. Therapist attitudes and patient outcomes: II. Therapist attitudes influence change during treatment. Psychotherapy Research 17, 2 (2007), 196–204.
  83. Music recommender systems. Recommender systems handbook (2015), 453–492.
  84. Human–AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support. Nature Machine Intelligence 5, 1 (2023), 46–57.
  85. Current challenges and barriers to real-world artificial intelligence adoption for the healthcare system, provider, and the patient. Translational Vision Science & Technology 9, 2 (2020), 45–45.
  86. Clay Spinuzzi. 2005. The methodology of participatory design. Technical communication 52, 2 (2005), 163–174.
  87. Malgorzata Monika Stanczyk. 2011. Music therapy in supportive cancer care. Reports of Practical Oncology & Radiotherapy 16, 5 (2011), 170–172.
  88. Emotion modulation through music after sadness induction—The iso principle in a controlled experimental study. International journal of environmental research and public health 18, 23 (2021), 12486.
  89. Teamind. 2022. https://www.teamind.co/
  90. Designing a musical fabric-based surface to encourage children with Autism to practice motor movements. In Proceedings of the 6th mexican conference on human-computer interaction. 1–4.
  91. Designing transparency for effective human-AI collaboration. Information Systems Frontiers 24, 3 (2022), 877–895.
  92. Bruce E Wampold and Christoph Flückiger. 2023. The alliance in mental health care: Conceptualization, evidence and clinical applications. World Psychiatry 22, 1 (2023), 25–41.
  93. “Brilliant AI doctor” in rural clinics: Challenges in AI-powered clinical decision support system deployment. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–18.
  94. Jeanne C Watson and Leslie S Greenberg. 2017. Emotion-focused therapy for generalized anxiety. American Psychological Association.
  95. Barbara L Wheeler. 2015. Music therapy handbook. Guilford Publications.
  96. WHO. 2022. COVID-19 pandemic triggers 25% increase in prevalence of anxiety and depression worldwide @OTHER. https://www.who.int/news/item/02-03-2022-covid-19-pandemic-triggers-25-increase-in-prevalence-of-anxiety-and-depression-worldwide
  97. On the use of ai for generation of functional music to improve mental health. Frontiers in Artificial Intelligence 3 (2020), 497864.
  98. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 CHI Conference on Human factors in computing systems. 1–13.
  99. Application and Innovation of Five Elements Music Therapy in the Era of Artificial Intelligence. In Proceedings of the 2nd International Symposium on Artificial Intelligence for Medicine Sciences. 132–136.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Jingjing Sun (8 papers)
  2. Jingyi Yang (19 papers)
  3. Yucheng Jin (11 papers)
  4. Jiangtao Gong (29 papers)
  5. Guyue Zhou (68 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets