ReXplain: Translating Radiology into Patient-Friendly Video Reports (2410.00441v2)

Abstract: Radiology reports, designed for efficient communication between medical experts, often remain incomprehensible to patients. This inaccessibility could potentially lead to anxiety, decreased engagement in treatment decisions, and poorer health outcomes, undermining patient-centered care. We present ReXplain (Radiology eXplanation), an innovative AI-driven system that translates radiology findings into patient-friendly video reports. ReXplain uniquely integrates a LLM for medical text simplification and text-anatomy association, an image segmentation model for anatomical region identification, and an avatar generation tool for engaging interface visualization. ReXplain enables producing comprehensive explanations with plain language, highlighted imagery, and 3D organ renderings in the form of video reports. To evaluate the utility of ReXplain-generated explanations, we conducted two rounds of user feedback collection from six board-certified radiologists. The results of this proof-of-concept study indicate that ReXplain could accurately deliver radiological information and effectively simulate one-on-one consultation, shedding light on enhancing patient-centered radiology with potential clinical usage. This work demonstrates a new paradigm in AI-assisted medical communication, potentially improving patient engagement and satisfaction in radiology care, and opens new avenues for research in multimodal medical communication.

• The paper introduces ReXplain, an AI system that converts complex radiology reports into patient-friendly video narratives.
• It integrates language simplification, image segmentation, and avatar narration to replicate radiologist consultations.
• Evaluation by radiologists showed improved patient understanding and engagement, indicating the system’s potential in advancing patient-centered care.

ReXplain: Translating Radiology into Patient-Friendly Video Reports

The paper presents ReXplain, an AI-driven system designed to convert complex radiology reports into accessible video reports for patients. This system seeks to address the gap in understanding that often exists between patients and their medical imaging results. ReXplain leverages advanced AI technologies to produce video reports that incorporate simplified language and multimodal content, simulating direct consultations with radiologists.

Core Components of ReXplain

ReXplain integrates multiple state-of-the-art AI models into a cohesive system, enabling the transformation of traditional radiology reports into video explanations:

• Language Simplification: Utilizes a LLM to translate detailed radiology reports into lay-language. This ensures that patients can comprehend the medical information without requiring any specialized knowledge.
• Image Segmentation: Employs a segmentation model to identify and highlight relevant anatomical regions within CT scans. This assists in visually connecting text descriptions to the corresponding images.
• Avatar Generation: Develops a virtual avatar to deliver the explanations. This feature is vital for creating an engaging and understandable presentation that mirrors a personalized consultation experience.

System Design and Implementation

ReXplain's design includes a pipeline that mimics the way radiologists would typically explain CT findings. Key stages involve:

1. Text Simplification: GPT-4o is employed to extract key findings from radiology reports and rephrase them into patient-friendly language.
2. Visual Localization: The selected findings are matched to anatomical structures, which are then segmented and highlighted using SAT.
3. Multimodal Presentation: The findings are presented alongside 3D renderings of organs and visual comparisons to normal images. This multimodal approach aims to facilitate better understanding.
4. Avatar-Based Narration: A virtual avatar generated by Tavus provides verbal explanations, enhancing the overall patient experience.

Evaluation and Insights

A proof-of-concept paper involving five board-certified radiologists assessed the effectiveness of ReXplain in clinical contexts. Key findings include:

• ReXplain's integration of text, imagery, and avatars was effective in translating medical jargon into easily digestible content for patients.
• The approach opens new possibilities for AI in healthcare communications, particularly in patient-centered radiology.
• Radiologists indicated that components such as lay-language explanations and image comparisons were particularly beneficial for patient comprehension.

Implications and Future Directions

The implementation of ReXplain signifies a novel step in bridging the communication gap between medical professionals and patients. By automating the translation of radiology reports into patient-friendly formats, ReXplain has the potential to enhance patient engagement and satisfaction significantly. Future research directions could focus on refining the accuracy of text-to-image associations, improving the realism of avatars, and expanding the system's applicability across various types of medical imaging.

Overall, this paper provides a valuable contribution to the field of medical informatics by demonstrating how AI can be seamlessly integrated into clinical workflows to improve patient outcomes. ReXplain stands as a promising tool for advancing patient-centered care in radiology, setting the stage for further exploration and development in AI-driven healthcare communication systems.