Interactive and Explainable Region-guided Radiology Report Generation (2304.08295v1)

Abstract: The automatic generation of radiology reports has the potential to assist radiologists in the time-consuming task of report writing. Existing methods generate the full report from image-level features, failing to explicitly focus on anatomical regions in the image. We propose a simple yet effective region-guided report generation model that detects anatomical regions and then describes individual, salient regions to form the final report. While previous methods generate reports without the possibility of human intervention and with limited explainability, our method opens up novel clinical use cases through additional interactive capabilities and introduces a high degree of transparency and explainability. Comprehensive experiments demonstrate our method's effectiveness in report generation, outperforming previous state-of-the-art models, and highlight its interactive capabilities. The code and checkpoints are available at https://github.com/ttanida/rgrg .

• The paper introduces the Region-Guided Radiology Report Generation (RGRG) method, using region-specific analysis instead of full images to improve explainability and interactivity.
• Experimental results on MIMIC-CXR demonstrate RGRG's ability to generate accurate reports with improved clinical efficacy and competitive text generation scores compared to other methods.
• The RGRG model offers interactivity by allowing users to query specific anatomical regions, making the AI-generated reports more transparent and adaptable for clinical use cases.

Interactive and Explainable Region-Guided Radiology Report Generation

The pursuit of automating radiology report generation is an endeavor driven by the need to ameliorate the workload of radiologists, given the high volume of imaging studies processed daily in clinical settings. The paper "Interactive and Explainable Region-guided Radiology Report Generation" presents a novel methodology that diverges from traditional full-image analysis techniques by introducing a more granular, region-specific approach. The proposed model, coined Region-Guided Radiology Report Generation (RGRG), employs an object detection mechanism to localize and analyze distinct anatomical regions within chest X-rays, facilitating the generation of concise, coherent report sections for each region.

Method Overview

The RGRG model is articulated as a four-pronged system, emphasizing detection, selection, and description of salient anatomical regions. Initially, the object detection component, based on Faster R-CNN with a ResNet-50 backbone, extracts visual properties of 29 predefined regions. This structured approach contrasts with previous methods that predominantly utilize holistic image features, potentially overlooking local anatomical variations.

Subsequent modules include a region selection mechanism and an abnormality classification unit, further refining which regions necessitate detailed descriptions. The region selection module addresses the challenge of identifying clinically relevant regions through binary classification, ensuring that only the most diagnostically pertinent regions are described. This mechanism aligns closely with a radiologist's decision-making process, in which selected regions bear higher pathological significance. The approach notably enhances both the explainability and adaptability of the tool within clinical workflows.

The LLM component, leveraging a fine-tuned GPT-2 architecture optimized with pseudo self-attention, generates detailed descriptions for the selected regions. This choice of architecture exploits the pre-training on medical abstracts to enrich the generated textual content, achieving factual completeness and clinical relevance.

Experimental Results

Experimental evaluations conducted on the MIMIC-CXR dataset provided substantial evidence of the model's efficacy. RGRG demonstrated proficiency in generating comprehensive and accurate reports as evidenced by improved METEOR scores and competitive BLEU-4 performance when compared to state-of-the-art systems. Noteworthy is the model's advancement in clinical efficacy metrics, with significant improvements in recall and F1 scores over baselines not directly optimized on these metrics. The model's ability to localize anatomically relevant regions and generate sentence-level analyses introduces a level of interactivity previously scarce in this domain.

The research explores the model's anatomy-based and selection-based sentence generation capabilities, affording clinicians the capacity to interrogate specific regions autonomously or through targeted bounding box annotations. This interactive dimension holds potential for integration into diagnostic radiology, offering customized reporting that aligns with varied clinical requirements.

Implications and Future Directions

The RGRG model's emphasis on region-specific processing and explainability heralds a meaningful step forward in the field of automated medical reporting. The method empowers radiologists with enhanced toolsets for validation and refinement of AI-generated content, fostering an environment of trust and safety crucial in healthcare applications. Future iterations could explore limited supervision scenarios to address the constraint of reliance on annotated datasets like Chest ImaGenome.

Practically, incorporating longitudinal analysis by referencing historical radiographs could mitigate current limitations observed in handling sequential imaging data. The expansion to integrate image-level features for capturing broader, non-localized pathologies will further enhance the model's breadth in capturing the full diagnostic picture.

In conclusion, the RGRG model presents a sophisticated, explainable strategy for radiology report generation, with interactive prospects critical to its success in clinical settings. This method stands poised to augment radiologist workflows, ensuring both adherence to accuracy and alignment with human oversight in diagnostic radiology.