Neural deformation fields for template-based reconstruction of cortical surfaces from MRI (2401.12938v2)
Abstract: The reconstruction of cortical surfaces is a prerequisite for quantitative analyses of the cerebral cortex in magnetic resonance imaging (MRI). Existing segmentation-based methods separate the surface registration from the surface extraction, which is computationally inefficient and prone to distortions. We introduce Vox2Cortex-Flow (V2C-Flow), a deep mesh-deformation technique that learns a deformation field from a brain template to the cortical surfaces of an MRI scan. To this end, we present a geometric neural network that models the deformation-describing ordinary differential equation in a continuous manner. The network architecture comprises convolutional and graph-convolutional layers, which allows it to work with images and meshes at the same time. V2C-Flow is not only very fast, requiring less than two seconds to infer all four cortical surfaces, but also establishes vertex-wise correspondences to the template during reconstruction. In addition, V2C-Flow is the first approach for cortex reconstruction that models white matter and pial surfaces jointly, therefore avoiding intersections between them. Our comprehensive experiments on internal and external test data demonstrate that V2C-Flow results in cortical surfaces that are state-of-the-art in terms of accuracy. Moreover, we show that the established correspondences are more consistent than in FreeSurfer and that they can directly be utilized for cortex parcellation and group analyses of cortical thickness.
- A lightweight approach to repairing digitized polygon meshes. The Visual Computer 26, 1393–1406. URL: http://link.springer.com/10.1007/s00371-010-0416-3, doi:10.1007/s00371-010-0416-3.
- Vox2cortex: Fast explicit reconstruction of cortical surfaces from 3d mri scans with geometric deep neural networks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20773–20783.
- Validation of FreeSurfer-Estimated Brain Cortical Thickness: Comparison with Histologic Measurements. Neuroinformatics 12, 535–542. doi:10.1007/s12021-014-9229-2.
- 3d u-net: Learning dense volumetric segmentation from sparse annotation, in: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016, Springer International Publishing, Cham. pp. 424–432.
- Meshlab: an open-source mesh processing tool. doi:10.2312/LOCALCHAPTEREVENTS/ITALCHAP/ITALIANCHAPCONF2008/129-136.
- Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage 9, 179–194.
- An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage 31, 968–980.
- Fast graph representation learning with PyTorch Geometric, in: ICLR Workshop on Representation Learning on Graphs and Manifolds.
- Freesurfer. Neuroimage 62, 774–781.
- Cortical surface-based analysis: Ii: inflation, flattening, and a surface-based coordinate system. Neuroimage 9, 195–207.
- High‐resolution inter subject averaging and a coordinate system for the cortical surface. Human Brain Mapping 8.
- Structural brain changes in aging: Courses, causes and cognitive consequences. Reviews in the Neurosciences 21, 187–222. doi:doi:10.1515/REVNEURO.2010.21.3.187.
- Surface simplification using quadric error metrics, in: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing Co., USA. p. 209–216. doi:10.1145/258734.258849.
- Mesh r-cnn, in: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9784–9794. doi:10.1109/ICCV.2019.00988.
- Segrecon: Learning joint brain surface reconstruction and segmentation from images , 650–659.
- AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation .
- 3d-coded: 3d correspondences by deep deformation, in: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.), Computer Vision – ECCV 2018, Springer International Publishing, Cham. pp. 235–251.
- Neural mesh flow: 3d manifold mesh generation via diffeomorphic flows, in: Proceedings of the 34th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA.
- Hypercolumns for object segmentation and fine-grained localization, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Boston, MA, USA. pp. 447–456. doi:10.1109/CVPR.2015.7298642.
- Fastsurfer - a fast and accurate deep learning based neuroimaging pipeline. NeuroImage 219, 117012. doi:https://doi.org/10.1016/j.neuroimage.2020.117012.
- Topofit: Rapid reconstruction of topologically-correct cortical surfaces .
- 3d whole brain segmentation using spatially localized atlas network tiles. NeuroImage 194, 105–119. doi:https://doi.org/10.1016/j.neuroimage.2019.03.041.
- Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: Proceedings of the 32nd International Conference on Machine Learning, PMLR, Lille, France. pp. 448–456.
- nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods .
- Japanese alzheimer's disease neuroimaging initiative: Present status and future. Alzheimer's & Dementia 6, 297–299. doi:10.1016/j.jalz.2010.03.011.
- 101 labeled brain images and a consistent human cortical labeling protocol. Frontiers in Neuroscience 6. doi:10.3389/fnins.2012.00171.
- A deep-learning approach for direct whole-heart mesh reconstruction. Medical Image Analysis 74, 102222. doi:https://doi.org/10.1016/j.media.2021.102222.
- Regionally localized thinning of the cerebral cortex in schizophrenia. Schizophrenia Research 60, 199–200.
- Corticalflow: A diffeomorphic mesh transformer network for cortical surface reconstruction, in: Advances in Neural Information Processing Systems.
- Focal decline of cortical thickness in alzheimer’s disease identified by computational neuroanatomy. Cerebral cortex 15 7, 995–1001.
- Efficient implementation of marching cubes’ cases with topological guarantees. Journal of graphics tools 8, 1–15.
- Marching cubes: A high resolution 3d surface construction algorithm, in: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, Association for Computing Machinery, New York, NY, USA. p. 163–169. doi:10.1145/37401.37422.
- Decoupled weight decay regularization, in: International Conference on Learning Representations.
- Cortexode: Learning cortical surface reconstruction by neural odes. IEEE Transactions on Medical Imaging , 1–1doi:10.1109/TMI.2022.3206221.
- PialNN: A Fast Deep Learning Framework for Cortical Pial Surface Reconstruction, in: Abdulkadir, A., Kia, S.M., Habes, M., Kumar, V., Rondina, J.M., Tax, C., Wolfers, T. (Eds.), Machine Learning in Clinical Neuroimaging. Springer International Publishing, Cham. volume 13001, pp. 73–81. doi:10.1007/978-3-030-87586-2_8.
- Reliability of brain volume measurements: A test-retest dataset. Scientific Data 1.
- Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults. Journal of Cognitive Neuroscience 19, 1498–1507. doi:10.1162/jocn.2007.19.9.1498.
- Occupancy networks: Learning 3d reconstruction in function space, in: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
- Discrete differential-geometry operators for triangulated 2-manifolds, in: Visualization and Mathematics III, Springer Berlin Heidelberg, Berlin, Heidelberg. pp. 35–57.
- Automatic segmentation of mr brain images with a convolutional neural network. IEEE Transactions on Medical Imaging 35, 1252–1261. doi:10.1109/TMI.2016.2548501.
- Weisfeiler and leman go neural: higher-order graph neural networks, in: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, AAAI Press, Honolulu, Hawaii, USA. pp. 4602–4609. doi:10.1609/aaai.v33i01.33014602.
- Rectified linear units improve restricted boltzmann machines, Omnipress, Madison, WI, USA. p. 807–814.
- Deepsdf: Learning continuous signed distance functions for shape representation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165–174.
- Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems 32. Curran Associates, Inc., Red Hook, NY, USA, pp. 8024–8035.
- Postmortem validation of MRI cortical volume measurements in MS. Human Brain Mapping 37, 2223–2233. doi:10.1002/hbm.23168.
- Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501 .
- U-net: Convolutional networks for biomedical image segmentation, in: MICCAI.
- Quicknat: A fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage 186, 713–727.
- Are 2.5d approaches superior to 3d deep networks in whole brain segmentation?, in: Medical Imaging with Deep Learning.
- Deepcsr: A 3d deep learning approach for cortical surface reconstruction. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , 806–815.
- Corticalflow++: Boosting cortical surface reconstruction accuracy, regularity, and interoperability, in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part V, Springer-Verlag, Berlin, Heidelberg. p. 496–505. doi:10.1007/978-3-031-16443-9_48.
- A large-scale comparison of cortical thickness and volume methods for measuring Alzheimer’s disease severity. NeuroImage: Clinical 11, 802–812. doi:10.1016/j.nicl.2016.05.017.
- Brainsuite: An automated cortical surface identification tool. Medical Image Anal. 6, 129–142. doi:10.1016/S1361-8415(02)00054-3.
- Age-related cortical thinning in cognitively healthy individuals in their 60s: the path through life study. Neurobiology of Aging 39, 202–209. doi:https://doi.org/10.1016/j.neurobiolaging.2015.12.009.
- Reconstruction of the human cerebral cortex robust to white matter lesions: Method and validation. Human Brain Mapping 35, 3385–3401. URL: https://doi.org/10.1002/hbm.22409, doi:10.1002/hbm.22409.
- Implicit neural representations with periodic activation functions, in: Proceedings of the 34th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA.
- GEOMetrics: Exploiting geometric structure for graph-encoded objects, in: Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, California, USA. pp. 5866–5876.
- Cyclical learning rates for training neural networks. 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) , 464–472.
- Advances in functional and structural mr image analysis and implementation as fsl. Neuroimage 23, S208–S219.
- Improved Laplacian Smoothing of Noisy Surface Meshes. Computer Graphics Forum 18, 131–138. URL: https://onlinelibrary.wiley.com/doi/10.1111/1467-8659.00334, doi:10.1111/1467-8659.00334.
- DeepNAT: Deep convolutional neural network for segmenting neuroanatomy. NeuroImage 170, 434–445. doi:10.1016/j.neuroimage.2017.02.035.
- Pixel2mesh: Generating 3d mesh models from single rgb images, in: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.), Computer Vision – ECCV 2018, Springer International Publishing, Cham. pp. 55–71.
- 3dn: 3d deformation network .
- Pixel2mesh++: Multi-view 3d mesh generation via deformation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , 1042–1051.
- Voxel2mesh: 3d mesh model generation from volumetric data , 299–308.
- Disn: Deep implicit surface network for high-quality single-view 3d reconstruction, in: NeurIPS.
- Spherical demons: Fast diffeomorphic landmark-free surface registration. IEEE Transactions on Medical Imaging 29, 650–668. doi:10.1109/TMI.2009.2030797.
- Local cortical surface complexity maps from spherical harmonic reconstructions. NeuroImage 56, 961–973. doi:https://doi.org/10.1016/j.neuroimage.2011.02.007.
- 3d u-net with multi-level deep supervision: Fully automatic segmentation of proximal femur in 3d mr images, in: Machine Learning in Medical Imaging, Springer International Publishing, Cham. pp. 274–282.
- Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters 15, 749–753. doi:10.1109/LGRS.2018.2802944.