Papers
Topics
Authors
Recent
Search
2000 character limit reached

NYCTALE: Neuro-Evidence Transformer for Adaptive and Personalized Lung Nodule Invasiveness Prediction

Published 15 Feb 2024 in cs.CV, cs.LG, eess.IV, and cs.NE | (2402.10066v1)

Abstract: Drawing inspiration from the primate brain's intriguing evidence accumulation process, and guided by models from cognitive psychology and neuroscience, the paper introduces the NYCTALE framework, a neuro-inspired and evidence accumulation-based Transformer architecture. The proposed neuro-inspired NYCTALE offers a novel pathway in the domain of Personalized Medicine (PM) for lung cancer diagnosis. In nature, Nyctales are small owls known for their nocturnal behavior, hunting primarily during the darkness of night. The NYCTALE operates in a similarly vigilant manner, i.e., processing data in an evidence-based fashion and making predictions dynamically/adaptively. Distinct from conventional Computed Tomography (CT)-based Deep Learning (DL) models, the NYCTALE performs predictions only when sufficient amount of evidence is accumulated. In other words, instead of processing all or a pre-defined subset of CT slices, for each person, slices are provided one at a time. The NYCTALE framework then computes an evidence vector associated with contribution of each new CT image. A decision is made once the total accumulated evidence surpasses a specific threshold. Preliminary experimental analyses conducted using a challenging in-house dataset comprising 114 subjects. The results are noteworthy, suggesting that NYCTALE outperforms the benchmark accuracy even with approximately 60% less training data on this demanding and small dataset.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. F. Piccialli, V. Di Somma, F. Giampaolo, S. Cuomo, and G. Fortino, “A survey on deep learning in medicine: Why, how and when?” Information Fusion, vol. 66, pp. 111–137, 2021.
  2. K. B. Johnson, W.-Q. Wei, D. Weeraratne, M. E. Frisse, K. Misulis, K. Rhee, J. Zhao, and J. L. Snowdon, “Precision medicine, ai, and the future of personalized health care,” Clinical and translational science, vol. 14, no. 1, pp. 86–93, 2021.
  3. P. Afshar, A. Mohammadi, K. N. Plataniotis, A. Oikonomou, and H. Benali, “From handcrafted to deep-learning-based cancer radiomics: challenges and opportunities,” IEEE Signal Processing Magazine, vol. 36, no. 4, pp. 132–160, 2019.
  4. S. Zhang, S. M. H. Bamakan, Q. Qu, and S. Li, “Learning for personalized medicine: a comprehensive review from a deep learning perspective,” IEEE Reviews in Biomedical Engineering, vol. 12, 2018.
  5. Y. Yang, L. Xu, L. Sun, P. Zhang, and S. S. Farid, “Machine learning application in personalised lung cancer recurrence and survivability prediction,” Computational and Structural Biotechnology Journal, vol. 20, pp. 1811–1820, 2022.
  6. G. Z. Papadakis, A. H. Karantanas, M. Tsiknakis, A. Tsatsakis, D. A. Spandidos, and K. Marias, “Deep learning opens new horizons in personalized medicine,” Biomedical reports, vol. 10, no. 4, 2019.
  7. N. J. Schork, “Artificial intelligence and personalized medicine,” Precision medicine in Cancer therapy, pp. 265–283, 2019.
  8. R. Ratcliff, P. L. Smith, S. D. Brown, and G. McKoon, “Diffusion decision model: Current issues and history,” Trends in Cognitive Sciences, vol. 20, no. 4, pp. 260–281, 2016.
  9. R. Ratcliff and G. McKoon, “The diffusion decision model: theory and data for two-choice decision tasks,” Neural computation, vol. 20, no. 4, pp. 873–922, 2008.
  10. M. Tanaka, Á. Szabó, L. Vécsei, and L. Giménez-Llort, “Emerging translational research in neurological and psychiatric diseases: from in vitro to in vivo models,” p. 15739, 2023.
  11. Y. Zhang, A. Oikonomou, A. Wong, M. A. Haider, and F. Khalvati, “Radiomics-based prognosis analysis for non-small cell lung cancer,” Scientific reports, vol. 7, no. 1, p. 46349, 2017.
  12. K. C. Thandra, A. Barsouk, K. Saginala, J. S. Aluru, and A. Barsouk, “Epidemiology of lung cancer,” Contemporary Oncology/Współczesna Onkologia, vol. 25, no. 1, pp. 45–52, 2021.
  13. Z. Chen, C. M. Fillmore, P. S. Hammerman, C. F. Kim, and K.-K. Wong, “Non-small-cell lung cancers: a heterogeneous set of diseases,” Nature Reviews Cancer, vol. 14, no. 8, pp. 535–546, 2014.
  14. C. I. Henschke, D. F. Yankelevitz, R. Mirtcheva, G. McGuinness, D. McCauley, and O. S. Miettinen, “Ct screening for lung cancer: frequency and significance of part-solid and nonsolid nodules,” American Journal of Roentgenology, vol. 178, no. 5, pp. 1053–1057, 2002.
  15. P. Afshar, A. Oikonomou, K. N. Plataniotis, and A. Mohammadi, “Mdr-surv: A multi-scale deep learning-based radiomics for survival prediction in pulmonary malignancies,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
  16. S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” Advances in neural information processing systems, 2017.
  17. P. Afshar, F. Naderkhani, A. Oikonomou, M. J. Rafiee, A. Mohammadi, and K. N. Plataniotis, “Mixcaps: A capsule network-based mixture of experts for lung nodule malignancy prediction,” Pattern Recognition, vol. 116, p. 107942, 2021.
  18. P. Afshar, A. Mohammadi, and K. N. Plataniotis, “Bayescap: A bayesian approach to brain tumor classification using capsule networks,” IEEE Signal Processing Letters, vol. 27, pp. 2024–2028, 2020.
  19. P. Afshar, A. Oikonomou, F. Naderkhani, P. N. Tyrrell, K. N. Plataniotis, K. Farahani, and A. Mohammadi, “3d-mcn: a 3d multi-scale capsule network for lung nodule malignancy prediction,” Scientific reports, vol. 10, no. 1, p. 7948, 2020.
  20. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  21. S. Khademi, A. Oikonomou, K. N. Plataniotis, and A. Mohammadi, “Cross attention-based fusion of deep and radiomics features for lung nodule invasiveness prediction,” in 24th International Conference on Digital Signal Processing (DSP), 2023.
  22. S. Khademi, S. Heidarian, P. Afshar, F. Naderkhani, A. Oikonomou, K. N. Plataniotis, and A. Mohammadi, “Spatio-temporal hybrid fusion of cae and swin transformers for lung cancer malignancy prediction,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).   IEEE, 2023, pp. 1–5.
  23. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022.
  24. A. Agarwal, A. Kumar V, K. Dunovan, E. Peterson, T. Verstynen, and K. Sycara, “Better safe than sorry: Evidence accumulation allows for safe reinforcement learning,” arXiv preprint arXiv:1809.09147, 2018.
  25. S. Sharma, D. Hughes, and K. Sycara, “CBGT-Net: A neuromimetic architecture for robust classification of streaming data,” in IEEE Int. Conference on Human-Machine Systems (ICHMS), 2024.
  26. C.-H. Chen, C.-K. Chang, C.-Y. Tu, W.-C. Liao, B.-R. Wu, K.-T. Chou, Y.-R. Chiou, S.-N. Yang, G. Zhang, and T.-C. Huang, “Radiomic features analysis in computed tomography images of lung nodule classification,” PloS one, vol. 13, no. 2, p. e0192002, 2018.
  27. Y. Xu, L. Lu, L.-n. E, W. Lian, H. Yang, L. H. Schwartz, Z.-h. Yang, and B. Zhao, “Application of radiomics in predicting the malignancy of pulmonary nodules in different sizes,” American Journal of Roentgenology, vol. 213, no. 6, pp. 1213–1220, 2019.
  28. T. Shen, R. Hou, X. Ye, X. Li, J. Xiong, Q. Zhang, C. Zhang, X. Cai, W. Yu, J. Zhao et al., “Predicting malignancy and invasiveness of pulmonary subsolid nodules on ct images using deep learning,” Frontiers in Oncology, vol. 11, p. 700158, 2021.
  29. R. Li, C. Xiao, Y. Huang, H. Hassan, and B. Huang, “Deep learning applications in computed tomography images for pulmonary nodule detection and diagnosis: A review,” Diagnostics, vol. 12, no. 2, 2022.
  30. L. Wang, “Deep learning techniques to diagnose lung cancer,” Cancers, vol. 14, no. 22, p. 5569, 2022.
  31. K. Mehta, A. Jain, J. Mangalagiri, S. Menon, P. Nguyen, and D. R. Chapman, “Lung nodule classification using biomarkers, volumetric radiomics, and 3d cnns,” Journal of Digital Imaging, pp. 1–20, 2021.
  32. S. Li, P. Xu, B. Li, L. Chen, Z. Zhou, H. Hao, Y. Duan, M. Folkert, J. Ma, S. Huang et al., “Predicting lung nodule malignancies by combining deep convolutional neural network and handcrafted features,” Physics in Medicine & Biology, vol. 64, no. 17, p. 175012, 2019.
  33. M. M. Farhangi, N. Petrick, B. Sahiner, H. Frigui, A. A. Amini, and A. Pezeshk, “Recurrent attention network for false positive reduction in the detection of pulmonary nodules in thoracic ct scans,” Medical physics, vol. 47, no. 5, pp. 2150–2160, 2020.
  34. Y. Luo, W. Liu, T. Fang, Q. Song, X. Min, M. Wang, and A. Li, “Carl: Cross-aligned representation learning for multi-view lung cancer histology classification,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, H. Greenspan, A. Madabhushi, P. Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, and R. Taylor, Eds.   Springer Nature Switzerland, 2023, pp. 358–367.
  35. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
  36. S. Paul and P.-Y. Chen, “Vision transformers are robust learners,” in Proceedings of the AAAI conference on Artificial Intelligence, vol. 36, no. 2, 2022, pp. 2071–2081.
  37. N. J. Evans and E.-J. Wagenmakers, “Evidence accumulation models: Current limitations and future directions,” 2019.
  38. R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen, “The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks,” Psychological review, vol. 113, no. 4, p. 700, 2006.
  39. S. D. Brown and A. Heathcote, “The simplest complete model of choice response time: Linear ballistic accumulation,” Cognitive Psychology, vol. 57, no. 3, pp. 153–178, 2008.
  40. G. Venugopal and B. S. Raju, “Modelling grid navigation using reinforcement learning linear ballistic accumulators,” in 2023 International Joint Conference on Neural Networks (IJCNN), 2023, pp. 1–8.
  41. M. A. Pisauro, E. Fouragnan, C. Retzler, and M. G. Philiastides, “Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous eeg-fmri,” Nature communications, vol. 8, no. 1, p. 15808, 2017.
  42. M. Wang, B. Wu, P. A. Kirschner, and J. M. Spector, “Using cognitive mapping to foster deeper learning with complex problems in a computer-based environment,” Computers in Human Behavior, vol. 87, pp. 450–458, 2018.
  43. A. Oikonomou, P. Salazar, Y. Zhang, and et al., “Histogram-based models on non-thin section chest ct predict invasiveness of primary lung adenocarcinoma subsolid nodules,” Sci Rep, 2019.
  44. P. Pattanayak, E. Turkbey, and S. R, “Comparative evaluation of three software packages for liver and spleen segmentation and volumetry,” Academic Radiology, vol. 24, pp. 831–839, 2017.
  45. J. Deng, W. Dong, R. Socher, L. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255, 2009.
  46. M.  Saied, M.  Raafat, S. Yehia, and M. Khalil, “Efficient pulmonary nodules classification using radiomics and different artificial intelligence strategies,” Insights Imaging, 2023.
  47. S. Based Shuvo, “An automated end-to-end deep learning-based framework for lung cancer diagnosis by detecting and classifying the lung nodules,” arXiv e-prints, 2023.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up