Enhancing Interpretability of Vertebrae Fracture Grading using Human-interpretable Prototypes (2404.02830v2)
Abstract: Vertebral fracture grading classifies the severity of vertebral fractures, which is a challenging task in medical imaging and has recently attracted Deep Learning (DL) models. Only a few works attempted to make such models human-interpretable despite the need for transparency and trustworthiness in critical use cases like DL-assisted medical diagnosis. Moreover, such models either rely on post-hoc methods or additional annotations. In this work, we propose a novel interpretable-by-design method, ProtoVerse, to find relevant sub-parts of vertebral fractures (prototypes) that reliably explain the model's decision in a human-understandable way. Specifically, we introduce a novel diversity-promoting loss to mitigate prototype repetitions in small datasets with intricate semantics. We have experimented with the VerSe'19 dataset and outperformed the existing prototype-based method. Further, our model provides superior interpretability against the post-hoc method. Importantly, expert radiologists validated the visual interpretability of our results, showing clinical applicability.
- Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging. Radiology: Artificial Intelligence, 3(6):e200267, 2021.
- On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation. PLOS ONE, 10(7):e0130140, July 2015. ISSN 1932-6203. .
- Worldwide prevalence and incidence of osteoporotic vertebral fractures. Osteoporosis International, 28:1531–1542, 2017.
- Compression fractures detection on ct. In Medical imaging 2017: computer-aided diagnosis, volume 10134, pages 1036–1043. SPIE, 2017.
- Intra and interobserver reliability and agreement of semiquantitative vertebral fracture assessment on chest computed tomography. PloS one, 8(8):e71204, 2013.
- Concept Learners for Few-Shot Learning. arXiv:2007.07375 [cs, stat], March 2021.
- Unreported vertebral body compression fractures at abdominal multidetector ct. Radiology, 268(1):120–126, 2013.
- On the applicability of prototypical part learning in medical images: breast masses classification using protopnet. In International Conference on Pattern Recognition, pages 539–557. Springer, 2022.
- Pixel-grounded prototypical part networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4768–4779, 2024.
- Risk of mortality following clinical fractures. Osteoporosis international, 11:556–561, 2000.
- Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 839–847, 2017. URL https://api.semanticscholar.org/CorpusID:13678776.
- This Looks Like That: Deep Learning for Interpretable Image Recognition. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
- 3d convolutional sequence to sequence model for vertebral compression fractures identification in ct. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23, pages 743–752. Springer, 2020.
- Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10265–10275, 2022.
- Interpretable Vertebral Fracture Diagnosis. In Mauricio Reyes, Pedro Henriques Abreu, and Jaime Cardoso, editors, Interpretability of Machine Intelligence in Medical Image Computing, Lecture Notes in Computer Science, pages 71–81, Cham, 2022a. Springer Nature Switzerland. ISBN 978-3-031-17976-1. .
- Interpretable vertebral fracture diagnosis. In Interpretability of Machine Intelligence in Medical Image Computing: 5th International Workshop, iMIMIC 2022, Held in Conjunction with MICCAI 2022, Singapore, Singapore, September 22, 2022, Proceedings, pages 71–81. Springer, 2022b.
- Concept-based Explanation for Fine-grained Images and Its Application in Infectious Keratitis Classification. In Proceedings of the 28th ACM International Conference on Multimedia, MM ’20, pages 700–708, New York, NY, USA, October 2020. Association for Computing Machinery. ISBN 978-1-4503-7988-5. .
- Two-stream compare and contrast network for vertebral compression fracture diagnosis. IEEE Transactions on Medical Imaging, 40(9):2496–2506, 2021.
- Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8730–8738, June 2018. .
- Axiom-based grad-cam: Towards accurate visualization and explanation of cnns. ArXiv, abs/2008.02312, 2020. URL https://api.semanticscholar.org/CorpusID:221006223.
- This looks more like that: Enhancing Self-Explaining Models by Prototypical Relevance Propagation. arXiv:2108.12204 [cs], August 2021.
- Vertebral fracture assessment using a semiquantitative technique. Journal of bone and mineral research, 8(9):1137–1148, 1993.
- Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
- This looks like that… does it? shortcomings of latent space prototype interpretability in deep networks. arXiv preprint arXiv:2105.02968, 2021.
- Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
- Evaluation and improvement of interpretability for self-explainable part-prototype networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2011–2020, 2023.
- Grading Loss: A Fracture Grade-Based Metric Loss for Vertebral Fracture Detection. In Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, and Leo Joskowicz, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Lecture Notes in Computer Science, pages 733–742, Cham, 2020. Springer International Publishing. .
- Now You See Me (CME): Concept-based Model Extraction. page 15.
- Semantic Latent Space Regression of Diffusion Autoencoders for Vertebral Fracture Grading, March 2023.
- XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15714–15723, Nashville, TN, USA, June 2021. IEEE. ISBN 978-1-66544-509-2. .
- The (un) reliability of saliency methods. Explainable AI: Interpreting, explaining and visualizing deep learning, pages 267–280, 2019.
- Concept Bottleneck Models. arXiv:2007.04612 [cs, stat], December 2020.
- Deep Learning for Case-Based Reasoning Through Prototypes: A Neural Network That Explains Its Predictions. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), April 2018. ISSN 2374-3468.
- A vertebral segmentation dataset with fracture grading. Radiology: Artificial Intelligence, 2(4):e190138, 2020.
- Using ProtoPNet for Interpretable Alzheimer’s Disease Classification. Proceedings of the Canadian Conference on Artificial Intelligence, June 2021. .
- Neural Prototype Trees for Interpretable Fine-grained Image Recognition. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14928–14938, Nashville, TN, USA, June 2021. IEEE. ISBN 978-1-66544-509-2. .
- Understanding Neural Networks via Feature Visualization: A Survey. In Wojciech Samek, Grégoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Klaus-Robert Müller, editors, Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Lecture Notes in Computer Science, pages 55–76. Springer International Publishing, Cham, 2019. ISBN 978-3-030-28954-6. .
- Detection of vertebral fractures in ct using 3d convolutional neural networks. In Computational Methods and Clinical Applications for Spine Imaging: 6th International Workshop and Challenge, CSI 2019, Shenzhen, China, October 17, 2019, Proceedings 6, pages 3–14. Springer, 2020.
- Keypoints localization for joint vertebra detection and fracture severity quantification. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23, pages 723–732. Springer, 2020.
- Explainable artificial intelligence for breast tumour classification: Helpful or harmful. In International Workshop on Interpretability of Machine Intelligence in Medical Image Computing, pages 104–123. Springer, 2022.
- Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206–215, 2019.
- Interpretable Image Classification with Differentiable Prototypes Assignment, September 2022.
- Towards Explainable Artificial Intelligence. In Wojciech Samek, Grégoire Montavon, Andrea Vedaldi, Lars Kai Hansen, and Klaus-Robert Müller, editors, Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Lecture Notes in Computer Science, pages 5–22. Springer International Publishing, Cham, 2019. .
- Benchmarking saliency methods for chest x-ray interpretation. Nature Machine Intelligence, 4(10):867–878, 2022.
- Gesina Schwalbe. Concept Embedding Analysis: A Review. arXiv:2203.13909 [cs, stat], March 2022.
- Verse: a vertebrae labelling and segmentation benchmark for multi-detector ct images. Medical image analysis, 73:102166, 2021.
- Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
- Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
- These do not look like those: An interpretable deep learning model for image recognition. IEEE Access, 9:41482–41493, 2021.
- Towards human-interpretable prototypes for visual assessment of image classification models. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP, pages 878–887. INSTICC, SciTePress, 2023. ISBN 978-989-758-634-7. .
- Right for the wrong reason: Can interpretable ml techniques detect spurious correlations? In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 425–434. Springer, 2023.
- Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning, pages 3319–3328. PMLR, July 2017.
- A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI. arXiv:1907.07374 [cs], August 2020. .
- Deep neural networks for automatic detection of osteoporotic vertebral fractures on ct scans. Computers in biology and medicine, 98:8–15, 2018.
- Learning support and trivial prototypes for interpretable image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2062–2072, 2023.
- MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imaging, April 2023.
- Automated deep learning-based detection of osteoporotic fractures in ct images. In Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12, pages 376–385. Springer, 2021.
- Interpretable vertebral fracture quantification via anchor-free landmarks localization. Medical Image Analysis, 83:102646, 2023.
- Learning Deep Features for Discriminative Localization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2921–2929, June 2016. .