Boundary Constraint-free Biomechanical Model-Based Surface Matching for Intraoperative Liver Deformation Correction (2403.09964v2)
Abstract: In image-guided liver surgery, 3D-3D non-rigid registration methods play a crucial role in estimating the mapping between the preoperative model and the intraoperative surface represented as point clouds, addressing the challenge of tissue deformation. Typically, these methods incorporate a biomechanical model, represented as a finite element model (FEM), used to regularize a surface matching term. This paper introduces a novel 3D-3D non-rigid registration method. In contrast to the preceding techniques, our method uniquely incorporates the FEM within the surface matching term itself, ensuring that the estimated deformation maintains geometric consistency throughout the registration process. Additionally, we eliminate the need to determine zero-boundary conditions and applied force locations in the FEM. We achieve this by integrating soft springs into the stiffness matrix and allowing forces to be distributed across the entire liver surface. To further improve robustness, we introduce a regularization technique focused on the gradient of the force magnitudes. This regularization imposes spatial smoothness and helps prevent the overfitting of irregular noise in intraoperative data. Optimization is achieved through an accelerated proximal gradient algorithm, further enhanced by our proposed method for determining the optimal step size. Our method is evaluated and compared to both a learning-based method and a traditional method that features FEM regularization using data collected on our custom-developed phantom, as well as two publicly available datasets. Our method consistently outperforms or is comparable to the baseline techniques. Both the code and dataset will be made publicly available.
- J. S. Heiselman, W. R. Jarnagin, and M. I. Miga, “Intraoperative correction of liver deformation using sparse surface and vascular features via linearized iterative boundary reconstruction,” IEEE transactions on medical imaging, vol. 39, no. 6, pp. 2223–2234, 2020.
- J. N. Smit, K. F. Kuhlmann, B. R. Thomson, N. F. Kok, T. J. Ruers, and M. Fusaglia, “Ultrasound guidance in navigated liver surgery: toward deep-learning enhanced compensation of deformation and organ motion,” International journal of computer assisted radiology and surgery, pp. 1–9, 2023.
- I. Peterlik, H. Courtecuisse, R. Rohling, P. Abolmaesumi, C. Nguan, S. Cotin, and S. Salcudean, “Fast elastic registration of soft tissues under large deformations,” Medical image analysis, vol. 45, pp. 24–40, 2018.
- D. C. Rucker, Y. Wu, L. W. Clements, J. E. Ondrake, T. S. Pheiffer, A. L. Simpson, W. R. Jarnagin, and M. I. Miga, “A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data,” IEEE transactions on medical imaging, vol. 33, no. 1, pp. 147–158, 2013.
- S. Suwelack, S. Röhl, S. Bodenstedt, D. Reichard, R. Dillmann, T. dos Santos, L. Maier-Hein, M. Wagner, J. Wünscher, H. Kenngott, et al., “Physics-based shape matching for intraoperative image guidance,” Medical physics, vol. 41, no. 11, p. 111901, 2014.
- R. Modrzejewski, T. Collins, B. Seeliger, A. Bartoli, A. Hostettler, and J. Marescaux, “An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions,” International journal of computer assisted radiology and surgery, vol. 14, no. 7, pp. 1237–1245, 2019.
- A. Mendizabal, E. Tagliabue, and D. Dall’Alba, “Intraoperative estimation of liver boundary conditions from multiple partial surfaces,” International Journal of Computer Assisted Radiology and Surgery, pp. 1–8, 2023.
- Z. Yang, R. A. Simon, and C. A. Linte, “Disparity refinement framework for learning-based stereo matching methods in cross-domain setting for laparoscopic images,” Journal of Medical Imaging, vol. 10, no. 4, p. 045001, 2023.
- J. S. Heiselman, L. W. Clements, J. A. Collins, J. A. Weis, A. L. Simpson, S. K. Geevarghese, T. P. Kingham, W. R. Jarnagin, and M. I. Miga, “Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery,” Journal of Medical Imaging, vol. 5, no. 2, pp. 021203–021203, 2018.
- D. Lee, W. H. Nam, J. Y. Lee, and J. B. Ra, “Non-rigid registration between 3d ultrasound and ct images of the liver based on intensity and gradient information,” Physics in Medicine & Biology, vol. 56, no. 1, p. 117, 2010.
- M. Labrunie, D. Pizarro, C. Tilmant, and A. Bartoli, “Automatic 3d/2d deformable registration in minimally invasive liver resection using a mesh recovery network,” in Medical Imaging with Deep Learning, 2023.
- Y. Espinel, L. Calvet, K. Botros, E. Buc, C. Tilmant, and A. Bartoli, “Using multiple images and contours for deformable 3d-2d registration of a preoperative ct in laparoscopic liver surgery,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 657–666, Springer, 2021.
- T. Collins, D. Pizarro, S. Gasparini, N. Bourdel, P. Chauvet, M. Canis, L. Calvet, and A. Bartoli, “Augmented reality guided laparoscopic surgery of the uterus,” IEEE Transactions on Medical Imaging, vol. 40, no. 1, pp. 371–380, 2020.
- B. Deng, Y. Yao, R. M. Dyke, and J. Zhang, “A survey of non-rigid 3d registration,” in Computer Graphics Forum, vol. 41, pp. 559–589, Wiley Online Library, 2022.
- J. Zhang, Y. Zhong, and C. Gu, “Deformable models for surgical simulation: a survey,” IEEE reviews in biomedical engineering, vol. 11, pp. 143–164, 2017.
- G. Mestdagh and S. Cotin, “An optimal control problem for elastic registration and force estimation in augmented surgery,” in Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VII, pp. 74–83, Springer, 2022.
- S. Khallaghi, C. A. Sánchez, A. Rasoulian, Y. Sun, F. Imani, A. Khojaste, O. Goksel, C. Romagnoli, H. Abdi, S. Chang, et al., “Biomechanically constrained surface registration: Application to mr-trus fusion for prostate interventions,” IEEE transactions on medical imaging, vol. 34, no. 11, pp. 2404–2414, 2015.
- A. Myronenko and X. Song, “Point set registration: Coherent point drift,” IEEE transactions on pattern analysis and machine intelligence, vol. 32, no. 12, pp. 2262–2275, 2010.
- G. Mestdagh, “An optimal control formulation for organ registration in augmented surgery,” 2022.
- M. Pfeiffer, C. Riediger, S. Leger, J.-P. Kühn, D. Seppelt, R.-T. Hoffmann, J. Weitz, and S. Speidel, “Non-rigid volume to surface registration using a data-driven biomechanical model,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 724–734, Springer, 2020.
- E. Tagliabue, D. Dall’Alba, M. Pfeiffer, M. Piccinelli, R. Marin, U. Castellani, S. Speidel, and P. Fiorini, “Data-driven intra-operative estimation of anatomical attachments for autonomous tissue dissection,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 1856–1863, 2021.
- E. L. Brewer, L. W. Clements, J. A. Collins, D. J. Doss, J. S. Heiselman, M. I. Miga, C. D. Pavas, and E. H. Wisdom III, “The image-to-physical liver registration sparse data challenge,” in Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 10951, pp. 364–370, SPIE, 2019.
- B. Acidi, M. Ghallab, S. Cotin, E. Vibert, and N. Golse, “Augmented reality in liver surgery, where we stand in 2023,” Journal of Visceral Surgery, 2023.
- N. Golse, A. Petit, M. Lewin, E. Vibert, and S. Cotin, “Augmented reality during open liver surgery using a markerless non-rigid registration system,” Journal of Gastrointestinal Surgery, vol. 25, pp. 662–671, 2021.
- P. J. Besl and N. D. McKay, “Method for registration of 3-d shapes,” in Sensor fusion IV: control paradigms and data structures, vol. 1611, pp. 586–606, Spie, 1992.
- L. W. Clements, W. C. Chapman, B. M. Dawant, R. L. Galloway Jr, and M. I. Miga, “Robust surface registration using salient anatomical features for image-guided liver surgery: algorithm and validation,” Medical physics, vol. 35, no. 6Part1, pp. 2528–2540, 2008.
- J. Bonet and R. D. Wood, Nonlinear continuum mechanics for finite element analysis. Cambridge university press, 1997.
- Y. Nesterov, “A method for unconstrained convex minimization problem with the rate of convergence,” 1983.
- Z. Yang, R. Simon, and C. A. Linte, “Learning feature descriptors for pre-and intra-operative point cloud matching for laparoscopic liver registration,” International Journal of Computer Assisted Radiology and Surgery, pp. 1–8, 2023.
- M. R. Robu, J. Ramalhinho, S. Thompson, K. Gurusamy, B. Davidson, D. Hawkes, D. Stoyanov, and M. J. Clarkson, “Global rigid registration of ct to video in laparoscopic liver surgery,” International Journal of Computer Assisted Radiology and Surgery, vol. 13, no. 6, pp. 947–956, 2018.
- N. Parikh and S. Boyd, “Proximal algorithms,” Foundations and trends® in Optimization, vol. 1, no. 3, pp. 127–239, 2014.
- H. Jeffreys and B. Jeffreys, Methods of mathematical physics. Cambridge university press, 1999.
- “Use soft spring to stabilize model.” https://help.solidworks.com/2021/english/SolidWorks/cworks/c_Use_Soft_Spring_to_Stabilize_Model.htm, 2021.
- H. Kenngott, J. Wünscher, M. Wagner, A. Preukschas, A. Wekerle, P. Neher, S. Suwelack, S. Speidel, F. Nickel, D. Oladokun, et al., “Openhelp (heidelberg laparoscopy phantom): development of an open-source surgical evaluation and training tool,” Surgical endoscopy, vol. 29, pp. 3338–3347, 2015.
- S. Hang, “Tetgen, a delaunay-based quality tetrahedral mesh generator,” ACM Trans. Math. Softw, vol. 41, no. 2, p. 11, 2015.
- Q.-Y. Zhou, J. Park, and V. Koltun, “Open3d: A modern library for 3d data processing,” arXiv preprint arXiv:1801.09847, 2018.
- J. A. Collins, J. A. Weis, J. S. Heiselman, L. W. Clements, A. L. Simpson, W. R. Jarnagin, and M. I. Miga, “Improving registration robustness for image-guided liver surgery in a novel human-to-phantom data framework,” IEEE transactions on medical imaging, vol. 36, no. 7, pp. 1502–1510, 2017.
- G. Guennebaud, B. Jacob, et al., “Eigen v3.” http://eigen.tuxfamily.org, 2010.
- J. L. Sparks and R. B. Dupaix, “Constitutive modeling of rate-dependent stress–strain behavior of human liver in blunt impact loading,” Annals of biomedical engineering, vol. 36, pp. 1883–1892, 2008.