Exploring SDOH Influence on Liver Transplant Decisions Using LLMs
This paper presents a sophisticated framework that utilizes LLMs to extract and analyze the influence of social determinants of health (SDOH) on liver transplantation (LT) decisions. The framework addresses the significant challenge in healthcare where unstructured documentation such as clinical notes limits the quantification and understanding of SDOH impact on health outcomes and access, particularly in complex fields such as organ transplantation.
Study Overview
The primary objective is to uncover how SDOH affects the eligibility and decision-making process for liver transplant patients. The researchers developed an AI-driven framework capable of extracting 23 predefined SDOH factors from psychosocial evaluation notes of patients considered for liver transplantation. This extraction provides what the authors term as "SDOH snapshots," which significantly enhance the prediction accuracy of patient progression through transplantation evaluation stages.
Methodology
A retrospective analysis was conducted using deidentified electronic medical record (EMR) data from a cohort of 4,431 patients evaluated for liver transplantation at the University of California, San Francisco, covering 2012 to 2023. The paper outlines the extensive application of an LLM (gpt-4-turbo-128k) to extract pertinent SDOH data from qualitative clinical notes. The extracted data were then utilized in conjunction with structured clinical and demographic data to improve the predictive models for LT recommendation and listing, employing Extreme Gradient Boosting for prediction. The LLM approach was carefully validated against manual annotations, achieving high accuracy across SDOH categories.
Key Findings
- Reliable Extraction: The LLMs demonstrated a robust ability to extract reliable SDOH factors comparable to the manual efforts of seasoned clinicians, achieving an accuracy range of 0.70-0.98.
- Demographic Disparities: Significant variations in SDOH factors across different demographic groups were identified. For example, female patients and Indigenous/Pacific Islander patients exhibited higher prevalence in certain adverse SDOH factors.
- Temporal Trends: The paper revealed shifts over time in both demographic composition and SDOH factor prevalence, such as increasing documentation of mental health factors.
- Impact on Prediction Models: When integrated with clinical data, SDOH snapshots notably improved predictive power for both psychosocial recommendations (from 0.540 to 0.872 AUROC) and listing for transplantation (from 0.578 to 0.716 AUROC).
- Racial Disparities: The paper applied the Blinder-Oaxaca decomposition approach to quantify how much of the observed racial disparities in LT decisions could be explained by these factors, finding significant unexplained gaps, particularly for patients of unknown or unspecified race.
Implications and Future Directions
The implications of this research span both practical and theoretical realms. Practically, it highlights specific SDOH factors such as housing instability and social support that can be targeted for intervention to enhance equity and effectiveness in transplant care. Theoretically, the approach paves the way for integrating unstructured clinical information into decision-making processes across various medical domains.
The paper suggests potential for broad application in other organ transplant decisions or fields requiring comprehensive assessments of psychosocial factors. Furthermore, it urges caution in assuming the accuracy of documentation present in clinical notes, recognizing potential biases in data recording.
In conclusion, the paper represents a methodological advance in leveraging AI to understand complex decision-making processes impacted by SDOH, thus offering actionable insights for improving fairness and quality of care in liver transplantation and possibly beyond. The thoughtful discussion on limitations and future research directions ensures this framework's application can be refined and expanded upon as more sophisticated data analytic tools and AI models develop.