A Novel Collaborative Self-Supervised Learning Method for Radiomic Data (2302.09807v1)
Abstract: The computer-aided disease diagnosis from radiomic data is important in many medical applications. However, developing such a technique relies on annotating radiological images, which is a time-consuming, labor-intensive, and expensive process. In this work, we present the first novel collaborative self-supervised learning method to solve the challenge of insufficient labeled radiomic data, whose characteristics are different from text and image data. To achieve this, we present two collaborative pretext tasks that explore the latent pathological or biological relationships between regions of interest and the similarity and dissimilarity information between subjects. Our method collaboratively learns the robust latent feature representations from radiomic data in a self-supervised manner to reduce human annotation efforts, which benefits the disease diagnosis. We compared our proposed method with other state-of-the-art self-supervised learning methods on a simulation study and two independent datasets. Extensive experimental results demonstrated that our method outperforms other self-supervised learning methods on both classification and regression tasks. With further refinement, our method shows the potential advantage in automatic disease diagnosis with large-scale unlabeled data available.
- P. Lambin et al., “Radiomics: extracting more information from medical images using advanced feature analysis,” European journal of cancer, vol. 48, no. 4, pp. 441–446, 2012.
- R. GGillies, P. Kinahan, and H. Hricak, “Radiomics: Images are more than pictures,” They Are Data. Radiology, vol. 278, no. 2, pp. 563–577, 2016.
- A. Zwanenburg et al., “The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping,” Radiology, vol. 295, no. 2, p. 328, 2020.
- J. J. Van Griethuysen et al., “Computational radiomics system to decode the radiographic phenotype,” Cancer research, vol. 77, no. 21, pp. e104–e107, 2017.
- A. Zwanenburg et al., “Assessing robustness of radiomic features by image perturbation,” Scientific reports, vol. 9, no. 1, pp. 1–10, 2019.
- X. Ou et al., “Radiomics based on 18f-fdg pet/ct could differentiate breast carcinoma from breast lymphoma using machine-learning approach: A preliminary study,” Cancer medicine, vol. 9, no. 2, pp. 496–506, 2020.
- F. Isensee, P. Kickingereder, W. Wick, M. Bendszus, and K. H. Maier-Hein, “Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge,” in International MICCAI Brainlesion Workshop. Springer, 2017, pp. 287–297.
- S. S. Alahmari, D. Cherezov, D. B. Goldgof, L. O. Hall, R. J. Gillies, and M. B. Schabath, “Delta radiomics improves pulmonary nodule malignancy prediction in lung cancer screening,” Ieee Access, vol. 6, pp. 77 796–77 806, 2018.
- A. Conti, A. Duggento, I. Indovina, M. Guerrisi, and N. Toschi, “Radiomics in breast cancer classification and prediction,” in Seminars in cancer biology, vol. 72. Elsevier, 2021, pp. 238–250.
- P. Afshar, A. Mohammadi, K. N. Plataniotis, A. Oikonomou, and H. Benali, “From handcrafted to deep-learning-based cancer radiomics: challenges and opportunities,” IEEE Signal Processing Magazine, vol. 36, no. 4, pp. 132–160, 2019.
- Q. Feng and Z. Ding, “Mri radiomics classification and prediction in alzheimer’s disease and mild cognitive impairment: a review,” Current Alzheimer Research, vol. 17, no. 3, pp. 297–309, 2020.
- C. Salvatore, I. Castiglioni, and A. Cerasa, “Radiomics approach in the neurodegenerative brain,” Aging Clinical and Experimental Research, vol. 33, no. 6, pp. 1709–1711, 2021.
- L.-B. Cui et al., “Disease definition for schizophrenia by functional connectivity using radiomics strategy,” Schizophrenia bulletin, vol. 44, no. 5, pp. 1053–1059, 2018.
- Y. W. Park et al., “Differentiating patients with schizophrenia from healthy controls by hippocampal subfields using radiomics,” Schizophrenia Research, vol. 223, pp. 337–344, 2020.
- H. J. Park, B. Park, and S. S. Lee, “Radiomics and deep learning: hepatic applications,” Korean Journal of Radiology, vol. 21, no. 4, pp. 387–401, 2020.
- F. Valdora, N. Houssami, F. Rossi, M. Calabrese, and A. S. Tagliafico, “Rapid review: radiomics and breast cancer,” Breast cancer research and treatment, vol. 169, no. 2, pp. 217–229, 2018.
- L. He et al., “Machine learning prediction of liver stiffness using clinical and t2-weighted mri radiomic data,” American Journal of Roentgenology, vol. 213, no. 3, pp. 592–601, 2019.
- W. K. Jeong, N. Jamshidi, E. R. Felker, S. S. Raman, and D. S. Lu, “Radiomics and radiogenomics of primary liver cancers,” Clinical and molecular hepatology, vol. 25, no. 1, p. 21, 2019.
- N. Beig, K. Bera, and P. Tiwari, “Introduction to radiomics and radiogenomics in neuro-oncology: implications and challenges,” Neuro-oncology Advances, vol. 2, no. Supplement_4, pp. iv3–iv14, 2020.
- H. Li, N. A. Parikh, and L. He, “A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes,” Frontiers in neuroscience, vol. 12, p. 491, 2018.
- Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite bert for self-supervised learning of language representations,” arXiv preprint arXiv:1909.11942, 2019.
- T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning. PMLR, 2020, pp. 1597–1607.
- K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9729–9738.
- A. Kolesnikov, X. Zhai, and L. Beyer, “Revisiting self-supervised visual representation learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 1920–1929.
- X. Li, M. Jia, M. T. Islam, L. Yu, and L. Xing, “Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis,” IEEE Transactions on Medical Imaging, vol. 39, no. 12, pp. 4023–4033, 2020.
- D. Tomar, B. Bozorgtabar, M. Lortkipanidze, G. Vray, M. S. Rad, and J.-P. Thiran, “Self-supervised generative style transfer for one-shot medical image segmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2022, pp. 1998–2008.
- M. Nadif and F. Role, “Unsupervised and self-supervised deep learning approaches for biomedical text mining,” Briefings in Bioinformatics, vol. 22, no. 2, pp. 1592–1603, 2021.
- X. Li et al., “Rotation-oriented collaborative self-supervised learning for retinal disease diagnosis,” IEEE Transactions on Medical Imaging, vol. 40, no. 9, pp. 2284–2294, 2021.
- S. Ji, M. Hölttä, and P. Marttinen, “Does the magic of bert apply to medical code assignment? a quantitative study,” Computers in Biology and Medicine, vol. 139, p. 104998, 2021.
- X. Zhuang, Y. Li, Y. Hu, K. Ma, Y. Yang, and Y. Zheng, “Self-supervised feature learning for 3d medical images by playing a rubik’s cube,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 420–428.
- W. Bai et al., “Self-supervised learning for cardiac mr image segmentation by anatomical position prediction,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2019, pp. 541–549.
- Z. Feng, C. Xu, and D. Tao, “Self-supervised representation learning by rotation feature decoupling,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10 364–10 374.
- B. Peng et al., “A multilevel-roi-features-based machine learning method for detection of morphometric biomarkers in parkinson’s disease,” Neuroscience letters, vol. 651, pp. 88–94, 2017.
- P. Liu, P. Li, W. He, and L.-Q. Zhao, “Liver and spleen volume variations in patients with hepatic fibrosis,” World Journal of Gastroenterology: WJG, vol. 15, no. 26, p. 3298, 2009.
- D. K. Thompson et al., “Tracking regional brain growth up to age 13 in children born term and very preterm,” Nature communications, vol. 11, no. 1, pp. 1–11, 2020.
- A. Cajanus et al., “The association between distinct frontal brain volumes and behavioral symptoms in mild cognitive impairment, alzheimer’s disease, and frontotemporal dementia,” Frontiers in neurology, vol. 10, p. 1059, 2019.
- M. Bang et al., “An interpretable multiparametric radiomics model for the diagnosis of schizophrenia using magnetic resonance imaging of the corpus callosum,” Translational psychiatry, vol. 11, no. 1, pp. 1–8, 2021.
- M. P. Starmans, R. L. Miclea, S. R. Van Der Voort, W. J. Niessen, M. G. Thomeer, and S. Klein, “Classification of malignant and benign liver tumors using a radiomics approach,” in Medical Imaging 2018: Image Processing, vol. 10574. International Society for Optics and Photonics, 2018, p. 105741D.
- J. Wei et al., “Radiomics in liver diseases: Current progress and future opportunities,” Liver International, vol. 40, no. 9, pp. 2050–2063, 2020.
- H. Yue et al., “Machine learning-based ct radiomics method for predicting hospital stay in patients with pneumonia associated with sars-cov-2 infection: a multicenter study,” Annals of translational medicine, vol. 8, no. 14, 2020.
- L. Peng et al., “Distinguishing true progression from radionecrosis after stereotactic radiation therapy for brain metastases with machine learning and radiomics,” International Journal of Radiation Oncology* Biology* Physics, vol. 102, no. 4, pp. 1236–1243, 2018.
- L. Brunese, F. Mercaldo, A. Reginelli, and A. Santone, “An ensemble learning approach for brain cancer detection exploiting radiomic features,” Computer methods and programs in biomedicine, vol. 185, p. 105134, 2020.
- S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” arXiv preprint arXiv:1803.07728, 2018.
- M. Noroozi and P. Favaro, “Unsupervised learning of visual representations by solving jigsaw puzzles,” in European conference on computer vision. Springer, 2016, pp. 69–84.
- H. Spitzer, K. Kiwitz, K. Amunts, S. Harmeling, and T. Dickscheid, “Improving cytoarchitectonic segmentation of human brain areas with self-supervised siamese networks,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2018, pp. 663–671.
- X. Liu et al., “Self-supervised learning: Generative or contrastive,” IEEE Transactions on Knowledge and Data Engineering, 2021.
- D. Wu, H. Ren, and Q. Li, “Self-supervised dynamic ct perfusion image denoising with deep neural networks,” IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 5, no. 3, pp. 350–361, 2020.
- L. Chen, P. Bentley, K. Mori, K. Misawa, M. Fujiwara, and D. Rueckert, “Self-supervised learning for medical image analysis using image context restoration,” Medical image analysis, vol. 58, p. 101539, 2019.
- M. Ye, X. Zhang, P. C. Yuen, and S.-F. Chang, “Unsupervised embedding learning via invariant and spreading instance feature,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 6210–6219.
- J.-B. Grill et al., “Bootstrap your own latent-a new approach to self-supervised learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 21 271–21 284, 2020.
- A. Vaswani et al., “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
- K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 000–16 009.
- T. Saito and J.-I. Toriwaki, “New algorithms for euclidean distance transformation of an n-dimensional digitized picture with applications,” Pattern recognition, vol. 27, no. 11, pp. 1551–1565, 1994.
- L. Wang, Y. Zhang, and J. Feng, “On the euclidean distance of images,” IEEE transactions on pattern analysis and machine intelligence, vol. 27, no. 8, pp. 1334–1339, 2005.
- L. Bai and E. R. Hancock, “Graph kernels from the jensen-shannon divergence,” Journal of mathematical imaging and vision, vol. 47, no. 1, pp. 60–69, 2013.
- J. Lin, “Divergence measures based on the shannon entropy,” IEEE Transactions on Information theory, vol. 37, no. 1, pp. 145–151, 1991.
- S. Kullback and R. A. Leibler, “On information and sufficiency,” The annals of mathematical statistics, vol. 22, no. 1, pp. 79–86, 1951.
- L. M. Bregman, “The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming,” USSR computational mathematics and mathematical physics, vol. 7, no. 3, pp. 200–217, 1967.
- F. Corso et al., “The challenge of choosing the best classification method in radiomic analyses: Recommendations and applications to lung cancer ct images,” Cancers, vol. 13, no. 12, p. 3088, 2021.
- N. A. Parikh et al., “Perinatal risk and protective factors in the development of diffuse white matter abnormality on term-equivalent age magnetic resonance imaging in infants born very preterm,” The Journal of Pediatrics, vol. 233, pp. 58–65, 2021.
- I. S. Gousias et al., “Magnetic resonance imaging of the newborn brain: manual segmentation of labelled atlases in term-born and preterm infants,” Neuroimage, vol. 62, no. 3, pp. 1499–1509, 2012.
- A. Makropoulos et al., “The developing human connectome project: A minimal processing pipeline for neonatal cortical surface reconstruction,” Neuroimage, vol. 173, pp. 88–112, 2018.
- J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
- K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
- Y. W. Park et al., “Radiomics and machine learning may accurately predict the grade and histological subtype in meningiomas using conventional and diffusion tensor imaging,” European radiology, vol. 29, no. 8, pp. 4068–4076, 2019.
- M. Sollini et al., “Pet/ct radiomics in breast cancer: Mind the step,” Methods, vol. 188, pp. 122–132, 2021.
- C. Parmar, P. Grossmann, D. Rietveld, M. M. Rietbergen, P. Lambin, and H. J. Aerts, “Radiomic machine-learning classifiers for prognostic biomarkers of head and neck cancer,” Frontiers in oncology, vol. 5, p. 272, 2015.