CopilotCAD: Empowering Radiologists with Report Completion Models and Quantitative Evidence from Medical Image Foundation Models (2404.07424v1)
Abstract: Computer-aided diagnosis systems hold great promise to aid radiologists and clinicians in radiological clinical practice and enhance diagnostic accuracy and efficiency. However, the conventional systems primarily focus on delivering diagnostic results through text report generation or medical image classification, positioning them as standalone decision-makers rather than helpers and ignoring radiologists' expertise. This study introduces an innovative paradigm to create an assistive co-pilot system for empowering radiologists by leveraging LLMs and medical image analysis tools. Specifically, we develop a collaborative framework to integrate LLMs and quantitative medical image analysis results generated by foundation models with radiologists in the loop, achieving efficient and safe generation of radiology reports and effective utilization of computational power of AI and the expertise of medical professionals. This approach empowers radiologists to generate more precise and detailed diagnostic reports, enhancing patient outcomes while reducing the burnout of clinicians. Our methodology underscores the potential of AI as a supportive tool in medical diagnostics, promoting a harmonious integration of technology and human expertise to advance the field of radiology.
- Stable code instruct alpha. URL [https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b](https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b).
- Cancer imaging phenomics toolkit: quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome. Journal of Medical Imaging, 5(1):011018, 2018. doi: 10.1117/1.JMI.5.1.011018. URL https://doi.org/10.1117/1.JMI.5.1.011018.
- Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024.
- Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
- Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890, 2023.
- Wizardcoder: Empowering code large language models with evol-instruct, 2023.
- Segment anything in medical images. Nature Communications, 15:1–9, 2024.
- Lvm-med: Learning large-scale self-supervised vision models for medical imaging via second-order graph matching. arXiv preprint arXiv:2306.11925, 2023.
- Ct urography. American Journal of Roentgenology, 195(5):W320–W324, 2010. doi: 10.2214/AJR.10.4198. URL https://doi.org/10.2214/AJR.10.4198. PMID: 20966295.
- Code llama: Open foundation models for code, 2024.
- Large language models encode clinical knowledge. arXiv preprint arXiv:2212.13138, 2022.
- Gemma Team. Gemma: Open models based on gemini research and technology, 2024.
- Chatcad: Interactive computer-aided diagnosis on medical image using large language models. arXiv preprint arXiv:2302.07257, 2023.
- Totalsegmentator: Robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence, 5(5), 2023.
- Towards generalist foundation model for radiology. arXiv preprint arXiv:2308.02463, 2023.
- Tinyllama: An open-source small language model, 2024.
- Chatcad+: Towards a universal and reliable interactive cad using llms. arXiv preprint arXiv:2305.15964, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.