From Registration Uncertainty to Segmentation Uncertainty (2403.05111v1)
Abstract: Understanding the uncertainty inherent in deep learning-based image registration models has been an ongoing area of research. Existing methods have been developed to quantify both transformation and appearance uncertainties related to the registration process, elucidating areas where the model may exhibit ambiguity regarding the generated deformation. However, our study reveals that neither uncertainty effectively estimates the potential errors when the registration model is used for label propagation. Here, we propose a novel framework to concurrently estimate both the epistemic and aleatoric segmentation uncertainties for image registration. To this end, we implement a compact deep neural network (DNN) designed to transform the appearance discrepancy in the warping into aleatoric segmentation uncertainty by minimizing a negative log-likelihood loss function. Furthermore, we present epistemic segmentation uncertainty within the label propagation process as the entropy of the propagated labels. By introducing segmentation uncertainty along with existing methods for estimating registration uncertainty, we offer vital insights into the potential uncertainties at different stages of image registration. We validated our proposed framework using publicly available datasets, and the results prove that the segmentation uncertainties estimated with the proposed method correlate well with errors in label propagation, all while achieving superior registration performance.
- Z. Eaton-Rosen et al., “Towards safe deep learning: accurately quantifying biomarker uncertainty in neural network predictions,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I. Springer, 2018, pp. 691–699.
- A. Jungo and M. Reyes, “Assessing reliability and challenges of uncertainty estimations for medical image segmentation,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. Springer, 2019, pp. 48–56.
- X. Yang et al., “Quicksilver: Fast predictive image registration–a deep learning approach,” NeuroImage, vol. 158, pp. 378–396, 2017.
- J. Chen et al., “Transmorph: Transformer for unsupervised medical image registration,” Medical Image Analysis, vol. 82, pp. 102615, 2022.
- J. Chen et al., “A Survey on Deep Learning in Medical Image Registration: New Technologies, Uncertainty, Evaluation Metrics, and Beyond,” arXiv preprint arXiv:2307.15615, 2023.
- I. J. A. Simpson et al., “Probabilistic segmentation propagation from uncertainty in registration,” in Proccedings Medical Image Analysis and Understanding (MIUA), 2011.
- J. Chen et al., “Unsupervised learning of diffeomorphic image registration via transmorph,” in International Workshop on Biomedical Image Registration. Springer, 2022, pp. 96–102.
- Y. Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in International Conference on Machine Learning. PMLR, 2016, pp. 1050–1059.
- A. V. Dalca et al., “Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces,” Medical Image Analysis, vol. 57, pp. 226–236, 2019.
- J. Luo et al., “On the applicability of registration uncertainty,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. Springer, 2019, pp. 410–419.
- A. Kendall and Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?,” Advances in Neural Information Processing Systems, vol. 30, 2017.
- T. DeVries and G. W. Taylor, “Leveraging uncertainty estimates for predicting segmentation quality,” arXiv preprint arXiv:1807.00502, 2018.
- X. Gong et al., “Uncertainty learning towards unsupervised deformable medical image registration,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 2484–2493.
- M. Seitzer et al., “On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks,” in International Conference on Learning Representations, 2022.
- O. Bernard et al., “Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?,” IEEE Trans. Med. Imag., vol. 37, no. 11, pp. 2514–2525, 2018.
- V. M. Campello et al., “Multi-centre, multi-vendor and multi-disease cardiac segmentation: the m&ms challenge,” IEEE Trans. Med. Imag., vol. 40, no. 12, pp. 3543–3554, 2021.
- B. B. Avants et al., “Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain,” Medical Image Analysis, vol. 12, no. 1, pp. 26–41, 2008.
- G. Balakrishnan et al., “VoxelMorph: A learning framework for deformable medical image registration,” IEEE Trans. Med. Imag., vol. 38, no. 8, pp. 1788–1800, 2019.
- T. C. W. Mok and A. Chung, “Fast symmetric diffeomorphic image registration with convolutional neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4644–4653.
- Y. Liu et al., “On finite difference Jacobian computation in deformable image registration,” arXiv preprint arXiv:2212.06060, 2022.
- P. Aljabar et al., “Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy,” NeuroImage, vol. 46, no. 3, pp. 726–738, 2009.
- J. I. Gear et al., “EANM practical guidance on uncertainty analysis for molecular radiotherapy absorbed dose calculations,” Eur. J. Nucl. Med. Mol. Imaging, vol. 45, pp. 2456–2474, 2018.