Papers
Topics
Authors
Recent
Search
2000 character limit reached

Functional requirements to mitigate the Risk of Harm to Patients from Artificial Intelligence in Healthcare

Published 19 Sep 2023 in cs.AI | (2309.10424v1)

Abstract: The Directorate General for Parliamentary Research Services of the European Parliament has prepared a report to the Members of the European Parliament where they enumerate seven main risks of AI in medicine and healthcare: patient harm due to AI errors, misuse of medical AI tools, bias in AI and the perpetuation of existing inequities, lack of transparency, privacy and security issues, gaps in accountability, and obstacles in implementation. In this study, we propose fourteen functional requirements that AI systems may implement to reduce the risks associated with their medical purpose: AI passport, User management, Regulation check, Academic use only disclaimer, data quality assessment, Clinicians double check, Continuous performance evaluation, Audit trail, Continuous usability test, Review of retrospective/simulated cases, Bias check, eXplainable AI, Encryption and use of field-tested libraries, and Semantic interoperability. Our intention here is to provide specific high-level specifications of technical solutions to ensure continuous good performance and use of AI systems to benefit patients in compliance with the future EU regulatory framework.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. The European Commision. Proposal for a regulation of the european parliament and the council laying down harmonised rules artificial intelligence (artificial intelligence act) and amending certain union legislative, 2021.
  2. The European Commision. Proposal for a directive of the european parliament and the council on adapting non-contractual civil liability rules to artificial intelligence (ai liability directive), 2022/0303, 2022.
  3. The European Commision. Proposal for a directive of the european parliament and the council on liability for defective products, 2022.
  4. The proposed eu directives for ai liability leave worrying gaps likely to impact medical ai. NPJ Digital Medicine, 6(1):77, 2023.
  5. European Parliamentary Research Service. Artificial intelligence in healthcare: Applications, risks, and ethical and societal impacts, 2022.
  6. Carlos Sáez Silvestre. Probabilistic methods for multi-source and temporal biomedical data quality assessment. PhD thesis, Universitat Politècnica de València, 2016.
  7. User-centred design of a clinical decision support system for palliative care: Insights from healthcare professionals. Digital Health, 9:20552076221150735, 2023.
  8. John Brooke. Sus: a “quick and dirty’usability. Usability evaluation in industry, 189(3):189–194, 1996.
  9. James R Lewis. The system usability scale: past, present, and future. International Journal of Human–Computer Interaction, 34(7):577–590, 2018.
  10. Continual lifelong learning with neural networks: A review. Neural networks, 113:54–71, 2019.
  11. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: artificial intelligence, 2(3):e190043, 2020.
  12. Artificial intelligence in healthcare. Nature biomedical engineering, 2(10):719–731, 2018.
  13. The openehr foundation. Studies in health technology and informatics, 115:153–173, 2005.
  14. The hl7 clinical document architecture. Journal of the American Medical Informatics Association, 8(6):552–569, 2001.
  15. Patients with non-oncological chronic conditions: Improving end-of-life care through integrated care and early palliative care provision. International Journal of Integrated Care (IJIC), 19, 2019.
  16. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  17. The ethics of ai in health care: a mapping review. Social Science & Medicine, 260:113172, 2020.
  18. Big data and machine learning algorithms for health-care delivery. The Lancet Oncology, 20(5):e262–e273, 2019.
  19. The European Parliament and the Council of the European Union. Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation) (text with eea relevance)., 2016.
  20. I Glenn Cohen. Informed consent and medical artificial intelligence: What to tell the patient? Geo. LJ, 108:1425, 2019.
  21. Ethical and legal challenges of informed consent applying artificial intelligence in medical diagnostic consultations. AI & SOCIETY, 36:509–520, 2021.
  22. The European Parliament. European parliament resolution of 20 october 2020 with recommendations to the commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020.
  23. The global landscape of ai ethics guidelines. Nature machine intelligence, 1(9):389–399, 2019.
  24. Artificial intelligence for good health: a scoping review of the ethics literature. BMC medical ethics, 22(1):1–17, 2021.
  25. A governance model for the application of ai in health care. Journal of the American Medical Informatics Association, 27(3):491–497, 2020.
  26. Embedded ethics: a proposal for integrating ethics into the development of medical ai. BMC Medical Ethics, 23(1):6, 2022.
  27. Building the case for actionable ethics in digital health research supported by artificial intelligence. BMC medicine, 17(1):1–7, 2019.
  28. Boundaries between research ethics and ethical research use in artificial intelligence health research. Journal of Empirical Research on Human Research Ethics, 16(3):325–337, 2021.
  29. The challenges of big data for research ethics committees: A qualitative swiss study. Journal of Empirical Research on Human Research Ethics, 17(1-2):129–143, 2022.
  30. World Health Organization. Ethics and governance of artificial intelligence for healt, 2021.
  31. Unesco. Recommendation on the ethics of artificial intelligence, 2021.
  32. The aleph: A multi-purpose clinical decision support platform for palliative care screening. International Journal of Integrated Care, 22(S3), 2022.
  33. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
  34. The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 3(11):e745–e750, 2021.
  35. From development to deployment: dataset shift, causality, and shift-stable models in health ai. Biostatistics, 21(2):345–352, 2020.
  36. What hinders the uptake of computerized decision support systems in hospitals? a qualitative study and framework for implementation. Implementation Science, 12(1):1–13, 2017.
  37. A survey of clinicians on the use of artificial intelligence in ophthalmology, dermatology, radiology and radiation oncology. Scientific reports, 11(1):5193, 2021.
  38. Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464):447–453, 2019.
  39. Brian Pickering. Trust, but verify: informed consent, ai technologies, and public health emergencies. Future Internet, 13(5):132, 2021.
  40. Hidden in plain sight—reconsidering the use of race correction in clinical algorithms, 2020.
  41. S Alder. Ai company exposed 2.5 million patient records over the internet. HIPAA Journal, available at: https://www. hipaa journal. com/ai-company-exposed-2-5-million-patient-records-over-the-internet/(accessed 21 August 2020), 2020.
  42. Blay Whitby. Automating medicine the ethical way. In Machine medical ethics, pages 223–232. Springer, 2014.
  43. Closing the ai accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 33–44, 2020.
  44. Why digital medicine depends on interoperability. NPJ digital medicine, 2(1):79, 2019.
  45. The European Commision. European health data space. european commission public health, 2022.
  46. Artificial intelligence in medicine and healthcare: applications, availability and societal impact, 2020.
  47. The doctor-patient relationship with artificial intelligence. American Journal of Roentgenology, 212(2):308–310, 2019.
  48. Artificial intelligence in health care: The hope, the hype, the promise, the peril. Washington, DC: National Academy of Medicine, 10, 2019.
  49. A short guide for medical professionals in the era of artificial intelligence. NPJ digital medicine, 3(1):126, 2020.
  50. Anmol Arora. Conceptualising artificial intelligence as a digital healthcare innovation: an introductory review. Medical Devices: Evidence and Research, pages 223–230, 2020.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up