Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Real-World Performance of Autonomously Reporting Normal Chest Radiographs in NHS Trusts Using a Deep-Learning Algorithm on the GP Pathway (2306.16115v1)

Published 28 Jun 2023 in cs.CV

Abstract: AIM To analyse the performance of a deep-learning (DL) algorithm currently deployed as diagnostic decision support software in two NHS Trusts used to identify normal chest x-rays in active clinical pathways. MATERIALS AND METHODS A DL algorithm has been deployed in Somerset NHS Foundation Trust (SFT) since December 2022, and at Calderdale & Huddersfield NHS Foundation Trust (CHFT) since March 2023. The algorithm was developed and trained prior to deployment, and is used to assign abnormality scores to each GP-requested chest x-ray (CXR). The algorithm classifies a subset of examinations with the lowest abnormality scores as High Confidence Normal (HCN), and displays this result to the Trust. This two-site study includes 4,654 CXR continuous examinations processed by the algorithm over a six-week period. RESULTS When classifying 20.0% of assessed examinations (930) as HCN, the model classified exams with a negative predictive value (NPV) of 0.96. There were 0.77% of examinations (36) classified incorrectly as HCN, with none of the abnormalities considered clinically significant by auditing radiologists. The DL software maintained fast levels of service to clinicians, with results returned to Trusts in a mean time of 7.1 seconds. CONCLUSION The DL algorithm performs with a low rate of error and is highly effective as an automated diagnostic decision support tool, used to autonomously report a subset of CXRs as normal with high confidence. Removing 20% of all CXRs reduces workload for reporters and allows radiology departments to focus resources elsewhere.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (23)
  1. National Health Service. Diagnostic imaging dataset statistical release. NHS England and NHS Improvement, September 2020. Available at: https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2020/09/Provisional-Monthly-Diagnostic-Imaging-Dataset-Statistics-2020-09-24.pdf (accessed June 2023).
  2. National Health Service. Table 4 2021-22 bodysite provider counts 2022-12-15. Diagnostic Imaging Dataset 2021-22 Data, December 2022. Available at: https://www.england.nhs.uk/statistics/statistical-work-areas/diagnostic-imaging-dataset/diagnostic-imaging-dataset-2021-22-data/ (accessed June 2023).
  3. National Health Service. Diagnostic imaging dataset annual statistical release 2021/22. Diagnostic Imaging Dataset 2021-22 Data, December 2022. Available at: https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2022/12/Annual-Statistical-Release-2021-22-PDF-1.3-MB.pdf (accessed June 2023).
  4. British Medical Association. Nhs backlog data analysis. British Medical Association, May 2023. Available at: https://www.bma.org.uk/advice-and-support/nhs-delivery-and-workforce/pressures/nhs-backlog-data-analysis (accessed June 2023).
  5. National Health Service. Publication of rtt waiting times data crs. Consultant-led Referral to Treatment Waiting Times Data 2022-23, March 2023. Available at: https://www.england.nhs.uk/statistics/statistical-work-areas/rtt-waiting-times/rtt-data-2022-23/ (accessed June 2023).
  6. The Royal College of Radiologists. Clinical radiology uk workforce census 2020 report. Clinical Radiology, April 2021.
  7. The Royal College of Radiologists. Royal college of radiologists sounds alarm over jeopardising patient care due to planned cuts to diagnostic capacity to fund nhs pay rises. RCR, July 2022. Available at: https://www.rcr.ac.uk/posts/royal-college-radiologists-sounds-alarm-over-jeopardising-patient-care-due-planned-cuts (accessed June 2023).
  8. The Royal College of Radiologists. Nhs backlogs and waiting times inquiry. UK Parliament, November 2021. Available at: https://www.rcr.ac.uk/sites/default/files/rcr_submission_-_nhs_backlogs_and_waiting_times.pdf (accessed June 2023).
  9. Jarrel C Y Seah et al. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. The Lancet Digital Health, 3(8):e496–e506, 2021. https://doi.org/10.1016/S2589-7500(21)00106-0.
  10. Ivo Baltruschat et al. Comparison of deep learning approaches for multi-label chest x-ray classification. Scientific Reports, 9(1), 2019. https://doi.org/10.1038/s41598-019-42294-8.
  11. UK Government. The medical devices regulations 2002. UK Statutory Instruments, 2023. Available at: https://www.legislation.gov.uk/uksi/2002/618/contents/made (accessed June 2023).
  12. Medical devices: conformity assessment and the ukca mark. Health and social care, December 2020. Available at: https://www.gov.uk/guidance/medical-devices-conformity-assessment-and-the-ukca-mark (accessed June 2023).
  13. Tom Dyer et al. Diagnosis of normal chest radiographs using an autonomous deep-learning algorithm. Clinical Radiology, 76(6), 2021. https://doi.org/10.1016/j.crad.2021.01.015.
  14. Gao Huang et al. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  15. EfficientNet: Rethinking model scaling for convolutional neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 6105–6114. PMLR, 09–15 Jun 2019.
  16. Artificial intelligence to complement rather than replace radiologists in breast screening. The Lancet Digital Health, 4(7):e478–e479, 2022.
  17. Christian Leibig et al. Combining the strengths of radiologists and ai for breast cancer screening: a retrospective analysis. The Lancet Digital Health, 4(7):e507–e519, 2022.
  18. Vegard Antun et al. On instabilities of deep learning in image reconstruction and the potential costs of ai. In Proceedings of the National Academy of Sciences, volume 117, Dec 2020.
  19. Teaching yourself about structural racism will improve your machine learning. Biostatistics, October 2020.
  20. Judy Wawira Gichoya et al. Ai recognition of patient race in medical imaging: a modelling study. The Lancet Digital Health, 4(6):e406–e414, 2022.
  21. Explaining medical ai performance disparities across sites with confounder shapley value analysis. In Proceedings of Machine Learning Research, Nov 2021.
  22. Medicines & Healthcare products Regulatory Agency. Good machine learning practice for medical device development: Guiding principles. Good practice, inspections and enforcement, October 2021.
  23. Tom Dyer et al. Robustness of an artificial intelligence solution for diagnosis of normal chest x-rays. Preprint, 2022. Available at: https://arxiv.org/abs/2209.09204 (accessed June 2023).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now