Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Integrating Physiological Data with Large Language Models for Empathic Human-AI Interaction (2404.15351v1)

Published 14 Apr 2024 in eess.SP, cs.HC, and cs.LG

Abstract: This paper explores enhancing empathy in LLMs by integrating them with physiological data. We propose a physiological computing approach that includes developing deep learning models that use physiological data for recognizing psychological states and integrating the predicted states with LLMs for empathic interaction. We showcase the application of this approach in an Empathic LLM (EmLLM) chatbot for stress monitoring and control. We also discuss the results of a pilot study that evaluates this EmLLM chatbot based on its ability to accurately predict user stress, provide human-like responses, and assess the therapeutic alliance with the user.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. Emotional intelligence of large language models, Journal of Pacific Rim Psychology 17 (2023) 18344909231213958.
  2. Introducing wesad, a multimodal dataset for wearable stress and affect detection, in: Proceedings of the 20th ACM international conference on multimodal interaction, 2018, pp. 400–408.
  3. The falcon series of open language models, arXiv preprint arXiv:2311.16867 (2023).
  4. G. A. Gladstein, Understanding empathy: Integrating counseling, developmental, and social psychology perspectives., Journal of counseling psychology 30 (1983) 467.
  5. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (woebot): a randomized controlled trial, JMIR mental health 4 (2017) e7785.
  6. Efficacy of the digital therapeutic mobile app biobase to reduce stress and improve mental well-being among university students: randomized controlled trial, JMIR mHealth and uHealth 8 (2020) e17767.
  7. Using large language models in psychology, Nature Reviews Psychology 2 (2023) 688–701.
  8. Supporting the demand on mental health services with AI-based conversational large language models (LLMs), BioMedInformatics 4 (2023) 8–33.
  9. Multimodal llms for health grounded in individual-specific data, in: Workshop on Machine Learning for Multimodal Healthcare Data, Springer, 2023, pp. 86–102.
  10. Biosignal copilot: Leveraging the power of llms in drafting reports for biomedical signals, medRxiv (2023) 2023–06.
  11. S. H. Fairclough, Fundamentals of physiological computing, Interacting with computers 21 (2009) 133–145.
  12. Smart devices and wearable technologies to detect and monitor mental health conditions and stress: A systematic review, Sensors 21 (2021) 3461.
  13. A scoping review of sensors, wearables, and remote monitoring for behavioral health: uses, outcomes, clinical competencies, and research directions, Journal of Technology in Behavioral Science 6 (2021) 278–313.
  14. S. Gedam, S. Paul, A review on mental stress detection using wearable sensors and machine learning techniques, IEEE Access 9 (2021) 84045–84066.
  15. AffectivelyVR: Towards VR personalized emotion recognition, in: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology, 2020, pp. 1–3.
  16. Deep learning for time series classification: a review, Data mining and knowledge discovery 33 (2019) 917–963.
  17. Convolutional neural networks for time series classification, Journal of Systems Engineering and Electronics 28 (2017) 162–169.
  18. Deepsense: A unified deep learning framework for time-series mobile sensing data processing, in: Proceedings of the 26th international conference on world wide web, 2017, pp. 351–360.
  19. Attention-based lstm-cnns for time-series classification, in: Proceedings of the 16th ACM conference on embedded networked sensor systems, 2018, pp. 410–411.
  20. Time series classification using multi-channels deep convolutional neural networks, in: International conference on web-age information management, Springer, 2014, pp. 298–310.
  21. R. Caruana, Multitask learning, Machine learning 28 (1997) 41–75.
  22. A prompt pattern catalog to enhance prompt engineering with chatgpt, arXiv preprint arXiv:2302.11382 (2023).
  23. Chain of empathy: Enhancing empathetic response of large language models based on psychotherapy models, arXiv preprint arXiv:2311.04915 (2023).
  24. Qlora: Efficient finetuning of quantized llms, Advances in Neural Information Processing Systems 36 (2024).
  25. W. S. Helton, K. Näswall, Short stress state questionnaire, European Journal of Psychological Assessment (2015).
  26. Godspeed questionnaire series: Translations and usage, International Handbook of Behavioral Health Assessment (2023).
  27. The session rating scale: Preliminary psychometric properties of a “working” alliance measure, Journal of brief Therapy 3 (2003) 3–12.
  28. Systematic review and meta-analysis of depression, anxiety, and suicidal ideation among Ph. D. students, Scientific Reports 11 (2021) 14370.
  29. A conceptual framework for designing interactive human-centred building spaces to enhance user experience in specific-purpose buildings, arXiv preprint arXiv:2308.14876 (2023).
  30. S. Sonnentag, U.-V. Bayer, Switching off mentally: predictors and consequences of psychological detachment from work during off-job time., Journal of occupational health psychology 10 (2005) 393.
  31. S. H. Fairclough, Physiological computing and intelligent adaptation, in: Emotions and affect in human factors and human-computer interaction, Elsevier, 2017, pp. 539–556.
  32. S. H. Fairclough, K. Gilleade, Meaningful interaction with physiological computing, in: Advances in physiological computing, Springer, 2014, pp. 1–16.
  33. Privacy, safety, and security in extended reality: User experience challenges for neurodiverse users, in: International Conference on Human-Computer Interaction, Springer, 2023, pp. 511–528.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Poorvesh Dongre (4 papers)
  2. Majid Behravan (7 papers)
  3. Kunal Gupta (12 papers)
  4. Mark Billinghurst (11 papers)
  5. Denis Gračanin (25 papers)
Citations (3)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets