Papers
Topics
Authors
Recent
2000 character limit reached

CataractBot: An LLM-Powered Expert-in-the-Loop Chatbot for Cataract Patients (2402.04620v6)

Published 7 Feb 2024 in cs.HC and cs.LG

Abstract: The healthcare landscape is evolving, with patients seeking reliable information about their health conditions and available treatment options. Despite the abundance of information sources, the digital age overwhelms individuals with excess, often inaccurate information. Patients primarily trust medical professionals, highlighting the need for expert-endorsed health information. However, increased patient loads on experts has led to reduced communication time, impacting information sharing. To address this gap, we developed CataractBot. CataractBot answers cataract surgery related questions instantly using an LLM to query a curated knowledge base, and provides expert-verified responses asynchronously. It has multimodal and multilingual capabilities. In an in-the-wild deployment study with 49 patients and attendants, 4 doctors, and 2 patient coordinators, CataractBot demonstrated potential, providing anytime accessibility, saving time, accommodating diverse literacy levels, alleviating power differences, and adding a privacy layer between patients and doctors. Users reported that their trust in the system was established through expert verification. Broadly, our results could inform future work on expert-mediated LLM bots.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. [n. d.]. An AI service for real-time document and text translation. https://www.microsoft.com/en-us/ai/ai-translator Accessed: 2024-02-01.
  2. Technical metrics used to evaluate health care chatbots: scoping review. Journal of medical Internet research 22, 6 (2020), e18301.
  3. MEGA: Multilingual Evaluation of Generative AI. arXiv:2303.12528 [cs.CL]
  4. Patient information needs: pre- and post-consultation. Health Informatics Journal 12, 2 (2006), 165–177. https://doi.org/10.1177/1460458206063811 arXiv:https://doi.org/10.1177/1460458206063811 PMID: 17023406.
  5. AI chatbots not yet ready for clinical use. Frontiers in Digital Health 5 (2023), 60.
  6. Analee E. Beisecker and Thomas D. Beisecker. 1990. Patient Information-Seeking Behaviors When Communicating with Doctors. Medical Care 28, 1 (1990), 19–28. http://www.jstor.org/stable/3765618
  7. Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents. In Proceedings of the SIGCHI conference on human factors in computing systems. 1265–1274.
  8. Jaydeep Borkar. 2023. What can we learn from Data Leakage and Unlearning for Law? arXiv preprint arXiv:2307.10476 (2023).
  9. G Byrne and R Heyman. 1997. Patient anxiety in the accident and emergency department. J. Clin. Nurs. 6, 4 (July 1997), 289–295.
  10. Åsa Cajander and Christiane Grünloh. 2019a. Electronic Health Records Are More Than a Work Tool: Conflicting Needs of Direct and Indirect Stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300865
  11. Åsa Cajander and Christiane Grünloh. 2019b. Electronic Health Records Are More Than a Work Tool: Conflicting Needs of Direct and Indirect Stakeholders. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3290605.3300865
  12. Assessment of patient information needs: A systematic review of measures. PLOS ONE 14 (01 2019), e0209165. https://doi.org/10.1371/journal.pone.0209165
  13. Bringing the field into focus: user-centered design of a patient expertise locator. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 1675–1684. https://doi.org/10.1145/1753326.1753577
  14. Health information needs, sources, and barriers of primary care patients to achieve patient-centered care: A literature review. Health Informatics Journal 22, 4 (2016), 992–1016. https://doi.org/10.1177/1460458215602939 arXiv:https://doi.org/10.1177/1460458215602939 PMID: 26377952.
  15. Inger Dahlén and Christer Janson. 2002. Anxiety and depression are related to the outcome of emergency treatment in patients with obstructive pulmonary disease. Chest 122, 5 (Nov. 2002), 1633–1637.
  16. Assessing Competence in Communication and Interpersonal Skills: The Kalamazoo II Report. Academic medicine : journal of the Association of American Medical Colleges 79 (07 2004), 495–507. https://doi.org/10.1097/00001888-200406000-00002
  17. Patients’ expectations and experiences of being involved in their own care in the emergency department: A qualitative interview study. Journal of clinical nursing 30, 13-14 (2021), 1942–1952.
  18. Ahmed Fadhil. 2018. A conversational interface to improve medication adherence: towards AI support in patient’s treatment. arXiv preprint arXiv:1803.09844 (2018).
  19. Improving Provision of Preanesthetic Information Through Use of the Digital Conversational Agent “MyAnesth”: Prospective Observational Trial. J Med Internet Res 22, 12 (4 Dec 2020), e20455. https://doi.org/10.2196/20455
  20. Comparing Semiliterate and Illiterate Users’ Ability to Transition from Audio+text to Text-Only Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1751–1760. https://doi.org/10.1145/1518701.1518971
  21. FarmChat: A Conversational Agent to Answer Farmer Queries. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4, Article 170 (dec 2018), 22 pages. https://doi.org/10.1145/3287048
  22. Understanding the benefits and challenges of deploying conversational AI leveraging large language models for public health intervention. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
  23. General practitioners’ experiences of digital written patient dialogues: a pilot study using a mixed method. Journal of Primary Care & Community Health 11 (2020), 2150132720909656.
  24. Patients’ experiences with a digital primary health care concept using written dialogues: a pilot study. Journal of Primary Care & Community Health 11 (2020), 2150132720910564.
  25. Knowledge is not power for patients: A systematic review and thematic synthesis of patient-reported barriers and facilitators to shared decision making. Patient Education and Counseling 94, 3 (2014), 291–309. https://doi.org/10.1016/j.pec.2013.10.031
  26. Design and evaluation of Devanagari virtual keyboards for touch screen mobile phones. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. 323–332.
  27. Interventions before consultations for helping patients address their information needs. Cochrane Database Syst. Rev. 2010, 3 (July 2007), CD004565.
  28. Blowing in the Wind: Unanchored Patient Information Work during Cancer Care. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 193–202. https://doi.org/10.1145/1753326.1753355
  29. Blowing in the wind: unanchored patient information work during cancer care. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 193–202. https://doi.org/10.1145/1753326.1753355
  30. Exploring the Use of Large Language Models for Improving the Awareness of Mindfulness. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–7.
  31. Suzanne M. Kurtz. 2002. Doctor-Patient Communication: Principles and Practices. Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques 29, S2 (2002), S23–S29. https://doi.org/10.1017/S0317167100001906
  32. Meaningful Explanation Effect on User’s Trust in an AI Medical System: Designing Explanations for Non-Expert Users. ACM Trans. Interact. Intell. Syst. 13, 4, Article 30 (dec 2023), 39 pages. https://doi.org/10.1145/3631614
  33. Inspiring AWE: Transforming Clinic Waiting Rooms into Informal Learning Environments with Active Waiting Education. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173672
  34. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 9459–9474. https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf
  35. Constraints and Workarounds to Support Clinical Consultations in Synchronous Text-based Platforms. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (¡conf-loc¿, ¡city¿Hamburg¡/city¿, ¡country¿Germany¡/country¿, ¡/conf-loc¿) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 342, 17 pages. https://doi.org/10.1145/3544548.3581014
  36. Assessment of nurse–patient communication and patient satisfaction from nursing care. Nursing open 6, 3 (2019), 1189–1196.
  37. Understanding online health information seeking behavior of older adults: A social cognitive perspective. Frontiers in Public Health 11 (2023), 1147789.
  38. Rebecca Moden. 2022. Blank Page Syndrome And How To Beat It. Retrieved Sept 1, 2023 from https://www.ool.co.uk/blog/blank-page-syndrome-and-how-to-beat-it/
  39. Anouk Mols and Jason Pridmore. 2021. Always available via WhatsApp: Mapping everyday boundary work practices and privacy negotiations. Mobile Media & Communication 9, 3 (2021), 422–440. https://doi.org/10.1177/2050157920970582
  40. Data decentralisation of LLM-based chatbot systems in chronic disease self-management. In Proceedings of the 2023 ACM Conference on Information Technology for Social Good. 205–212.
  41. Ministry of Health National Health Authority and Government of India Family Welfare. 2020. National Digital Health Mission: Strategy Overview. https://www.niti.gov.in/sites/default/files/2023-02/ndhm_strategy_overview.pdf
  42. “It Cannot Do All of My Work”: Community Health Worker Perceptions of AI-Enabled Mobile Health Applications in Rural India. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 701, 20 pages. https://doi.org/10.1145/3411764.3445420
  43. GPT-4 Technical Report. arXiv:arXiv:2303.08774
  44. Impacts of Symptom Checkers for Laypersons’ Self-diagnosis on Physicians in Primary Care: Scoping Review. Journal of Medical Internet Research 25 (2023), e39219.
  45. Recipes for building an open-domain chatbot. arXiv preprint arXiv:2004.13637 (2020).
  46. Review of manual small-incision cataract surgery. Indian journal of ophthalmology 65, 12 (2017), 1281.
  47. Catalyzing Social Support for Breast Cancer Patients. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 173–182. https://doi.org/10.1145/1753326.1753353
  48. Uptake and patient and provider communication modality preferences of virtual visits in primary care: a retrospective cohort study in Canada. BMJ open 10, 7 (2020), e037064.
  49. Statista. 2022. Most used social media and messaging apps in India 2022. https://www.statista.com/statistics/1235799/india-most-used-social-media-and-messaging-apps/ Accessed: 2024-02-01.
  50. Paul C Tang and David Lansky. 2005. The missing link: bridging the patient-provider health information gap. Health Aff. (Millwood) 24, 5 (Sept. 2005), 1290–1295.
  51. P C Tang and C Newcomb. 1998. Informing patients: a guide for providing patient health information. J. Am. Med. Inform. Assoc. 5, 6 (Nov. 1998), 563–570.
  52. Query Formulation as an Information Retrieval Problem. Comput. J. 39, 4 (01 1996), 255–274. https://doi.org/10.1093/comjnl/39.4.255 arXiv:https://academic.oup.com/comjnl/article-pdf/39/4/255/1916704/390255.pdf
  53. Communication skills for patient-centered care : Research-based, easily learned techniques for medical interviews that benefit orthopaedic surgeons and their patients. Journal of Bone and Joint Surgery, American Volume 87 (2005), 652–658. https://api.semanticscholar.org/CorpusID:71035586
  54. Understanding the Benefits and Design of Chatbots to Meet the Healthcare Needs of Migrant Workers. 7, CSCW2, Article 315 (oct 2023), 34 pages. https://doi.org/10.1145/3610106
  55. Please Call the Specialism: Using WeChat to Support Patient Care in China. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376274
  56. “Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (¡conf-loc¿, ¡city¿Yokohama¡/city¿, ¡country¿Japan¡/country¿, ¡/conf-loc¿) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 697, 18 pages. https://doi.org/10.1145/3411764.3445432
  57. CASS: Towards Building a Social-Support Chatbot for Online Health Community. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 9 (apr 2021), 31 pages. https://doi.org/10.1145/3449083
  58. BigBlueBot: teaching strategies for successful human-agent interactions. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI ’19). Association for Computing Machinery, New York, NY, USA, 448–459. https://doi.org/10.1145/3301275.3302290
  59. Designing Patient-Centric Information Displays for Hospitals. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 2123–2132. https://doi.org/10.1145/1753326.1753650
  60. Michael Williams and Tami Moser. 2019. The art of coding and thematic exploration in qualitative research. International Management Review 15, 1 (2019), 45–55.
  61. Powering an AI Chatbot with Expert Sourcing to Support Credible Health Information Access. In Proceedings of the 28th International Conference on Intelligent User Interfaces (Sydney, NSW, Australia) (IUI ’23). Association for Computing Machinery, New York, NY, USA, 2–18. https://doi.org/10.1145/3581641.3584031
  62. Feedpal: Understanding Opportunities for Chatbots in Breastfeeding Education of Women in India. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 170 (nov 2019), 30 pages. https://doi.org/10.1145/3359272
  63. Talk2Care: Facilitating Asynchronous Patient-Provider Communication with Large-Language-Model. arXiv preprint arXiv:2309.09357 (2023). arXiv:2309.09357 [cs.CL]
  64. PwR: Exploring the Role of Representations in Conversational Programming. arXiv:2309.09495 [cs.HC]
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 3 tweets with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free