Robust deformable image registration using cycle-consistent implicit representations (2310.01934v1)
Abstract: Recent works in medical image registration have proposed the use of Implicit Neural Representations, demonstrating performance that rivals state-of-the-art learning-based methods. However, these implicit representations need to be optimized for each new image pair, which is a stochastic process that may fail to converge to a global minimum. To improve robustness, we propose a deformable registration method using pairs of cycle-consistent Implicit Neural Representations: each implicit representation is linked to a second implicit representation that estimates the opposite transformation, causing each network to act as a regularizer for its paired opposite. During inference, we generate multiple deformation estimates by numerically inverting the paired backward transformation and evaluating the consensus of the optimized pair. This consensus improves registration accuracy over using a single representation and results in a robust uncertainty metric that can be used for automatic quality control. We evaluate our method with a 4D lung CT dataset. The proposed cycle-consistent optimization method reduces the optimization failure rate from 2.4% to 0.0% compared to the current state-of-the-art. The proposed inference method improves landmark accuracy by 4.5% and the proposed uncertainty metric detects all instances where the registration method fails to converge to a correct solution. We verify the generalizability of these results to other data using a centerline propagation task in abdominal 4D MRI, where our method achieves a 46% improvement in propagation consistency compared with single-INR registration and demonstrates a strong correlation between the proposed uncertainty metric and registration accuracy.
- W. M. Wells III, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis, “Multi-modal volume registration by maximization of mutual information,” Medical image analysis, vol. 1, no. 1, pp. 35–51, 1996.
- H. Wang and A. A. Amini, “Cardiac motion and deformation recovery from MRI: a review,” IEEE transactions on medical imaging, vol. 31, no. 2, pp. 487–503, 2011.
- P. Castadot, X. Geets, J. A. Lee, N. Christian, and V. Grégoire, “Assessment by a deformable registration method of the volumetric and positional changes of target volumes and organs at risk in pharyngo-laryngeal tumors treated with concomitant chemo-radiation,” Radiotherapy and Oncology, vol. 95, no. 2, pp. 209–217, 2010.
- A. Narang, R. Bae, H. Hong, Y. Thomas, S. Surette, C. Cadieu, A. Chaudhry, R. P. Martin, P. M. McCarthy, D. S. Rubenson et al., “Utility of a deep-learning algorithm to guide novices to acquire echocardiograms for limited diagnostic use,” JAMA cardiology, vol. 6, no. 6, pp. 624–632, 2021.
- C. S. De Jonge, A. J. Smout, A. J. Nederveen, and J. Stoker, “Evaluation of gastrointestinal motility with MRI: advances, challenges and opportunities,” Neurogastroenterology & Motility, vol. 30, no. 1, 2018.
- J. He, S. L. Baxter, J. Xu, J. Xu, X. Zhou, and K. Zhang, “The practical implementation of artificial intelligence technologies in medicine,” Nature medicine, vol. 25, no. 1, pp. 30–36, 2019.
- B. D. De Vos, F. F. Berendsen, M. A. Viergever, H. Sokooti, M. Staring, and I. Išgum, “A deep learning framework for unsupervised affine and deformable image registration,” Medical image analysis, vol. 52, pp. 128–143, 2019.
- G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V. Dalca, “Voxelmorph: a learning framework for deformable medical image registration,” IEEE transactions on medical imaging, vol. 38, no. 8, pp. 1788–1800, 2019.
- G. Haskins, U. Kruger, and P. Yan, “Deep learning in medical image registration: a survey,” Machine Vision and Applications, vol. 31, no. 1, pp. 1–18, 2020.
- A. Hering, L. Hansen, T. C. Mok, A. C. Chung, H. Siebert, S. Häger, A. Lange, S. Kuckertz, S. Heldmann, W. Shao et al., “Learn2reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning,” IEEE Transactions on Medical Imaging, 2022.
- W. Zhu, Y. Huang, D. Xu, Z. Qian, W. Fan, and X. Xie, “Test-time training for deformable multi-scale image registration,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 13 618–13 625.
- J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 165–174.
- L. Liu, J. Gu, K. Zaw Lin, T.-S. Chua, and C. Theobalt, “Neural sparse voxel fields,” Advances in Neural Information Processing Systems, vol. 33, pp. 15 651–15 663, 2020.
- B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
- J. M. Wolterink, J. C. Zwienenberg, and C. Brune, “Implicit neural representations for deformable image registration,” in Medical Imaging with Deep Learning, 2021.
- V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” Advances in Neural Information Processing Systems, vol. 33, pp. 7462–7473, 2020.
- M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” Advances in Neural Information Processing Systems, vol. 33, pp. 7537–7547, 2020.
- Z. M. Baum, Y. Hu, and D. C. Barratt, “Meta-registration: learning test-time optimization for single-pair image registration,” in Advances in Simplifying Medical UltraSound. Springer, 2022, pp. 162–171.
- G. E. Christensen and H. J. Johnson, “Consistent image registration,” IEEE transactions on medical imaging, vol. 20, no. 7, pp. 568–582, 2001.
- B. Kim, J. Kim, J.-G. Lee, D. H. Kim, S. H. Park, and J. C. Ye, “Unsupervised deformable image registration using cycle-consistent cnn,” in Medical Image Computing and Computer Assisted Intervention. Springer, 2019, pp. 166–174.
- B. Kim, D. H. Kim, S. H. Park, J. Kim, J.-G. Lee, and J. C. Ye, “Cyclemorph: cycle consistent unsupervised deformable image registration,” Medical image analysis, vol. 71, p. 102036, 2021.
- R. Castillo, E. Castillo, R. Guerra, V. E. Johnson, T. McPhail, A. K. Garg, and T. Guerrero, “A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets,” Physics in Medicine & Biology, vol. 54, no. 7, p. 1849, 2009.
- C. S. de Jonge, A. Menys, K. L. van Rijn, A. J. Bredenoord, A. J. Nederveen, and J. Stoker, “Detecting the effects of a standardized meal challenge on small bowel motility with MRI in prepared and unprepared bowel,” Neurogastroenterology & Motility, vol. 31, no. 2, 2019.
- L. D. van Harten, C. S. de Jonge, K. J. Beek, J. Stoker, and I. Išgum, “Untangling and segmenting the small intestine in 3d cine-MRI using deep learning,” Medical Image Analysis, vol. 78, p. 102386, 2022.
- D. Rueckert, L. I. Sonoda, C. Hayes, D. L. Hill, M. O. Leach, and D. J. Hawkes, “Nonrigid registration using free-form deformations: application to breast mr images,” IEEE transactions on medical imaging, vol. 18, no. 8, pp. 712–721, 1999.
- L. K. Hansen and P. Salamon, “Neural network ensembles,” IEEE transactions on pattern analysis and machine intelligence, vol. 12, no. 10, pp. 993–1001, 1990.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
- J. Hofmanninger, F. Prayer, J. Pan, S. Röhrich, H. Prosch, and G. Langs, “Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem,” European Radiology Experimental, vol. 4, no. 1, pp. 1–13, 2020.
- A. Hering, S. Häger, J. Moltz, N. Lessmann, S. Heldmann, and B. van Ginneken, “Cnn-based lung ct registration with multiple anatomical constraints,” Medical Image Analysis, vol. 72, p. 102139, 2021.
- F. F. Berendsen, A. N. Kotte, M. A. Viergever, and J. P. Pluim, “Registration of organs with sliding interfaces and changing topologies,” in Medical Imaging 2014: Image Processing, vol. 9034. SPIE, 2014, pp. 95–101.
- M. P. Heinrich, H. Handels, and I. J. Simpson, “Estimating large lung motion in copd patients by symmetric regularised correspondence fields,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part II 18. Springer, 2015, pp. 338–345.
- J. Rühaak, T. Polzin, S. Heldmann, I. J. Simpson, H. Handels, J. Modersitzki, and M. P. Heinrich, “Estimation of large motion in lung ct by integrating regularized keypoint correspondences into dense deformable registration,” IEEE transactions on medical imaging, vol. 36, no. 8, pp. 1746–1757, 2017.
- B. D. de Vos, B. H. van der Velden, J. Sander, K. G. Gilhuijs, M. Staring, and I. Išgum, “Mutual information for unsupervised deep learning image registration,” in Medical Imaging 2020: Image Processing, vol. 11313. SPIE, 2020, pp. 155–161.