Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Non-negative Subspace Feature Representation for Few-shot Learning in Medical Imaging (2404.02656v2)

Published 3 Apr 2024 in cs.CV and cs.AI

Abstract: Unlike typical visual scene recognition domains, in which massive datasets are accessible to deep neural networks, medical image interpretations are often obstructed by the paucity of data. In this paper, we investigate the effectiveness of data-based few-shot learning in medical imaging by exploring different data attribute representations in a low-dimensional space. We introduce different types of non-negative matrix factorization (NMF) in few-shot learning, addressing the data scarcity issue in medical image classification. Extensive empirical studies are conducted in terms of validating the effectiveness of NMF, especially its supervised variants (e.g., discriminative NMF, and supervised and constrained NMF with sparseness), and the comparison with principal component analysis (PCA), i.e., the collaborative representation-based dimensionality reduction technique derived from eigenvectors. With 14 different datasets covering 11 distinct illness categories, thorough experimental results and comparison with related techniques demonstrate that NMF is a competitive alternative to PCA for few-shot learning in medical imaging, and the supervised NMF algorithms are more discriminative in the subspace with greater effectiveness. Furthermore, we show that the part-based representation of NMF, especially its supervised variants, is dramatically impactful in detecting lesion areas in medical imaging with limited samples.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. J. Kaplan, S. McCandlish, T. J. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” ArXiv, vol. abs/2001.08361, 2020.
  2. M. I. Razzak, S. Naz, and A. Zaib, “Deep learning for medical image processing: Overview, challenges and the future,” Classification in BioApps, pp. 323–350, 2018.
  3. A. Babayan, M. Erbey, D. Kumral, J. D. Reinelt, A. M. Reiter, J. Röbbig, H. L. Schaare, M. Uhlig, A. Anwander, P.-L. Bazin et al., “A mind-brain-body dataset of mri, eeg, cognition, emotion, and peripheral physiology in young and old adults,” Scientific data, vol. 6, no. 1, pp. 1–21, 2019.
  4. K. Yan, X. Wang, L. Lu, and R. M. Summers, “Deeplesion: Automated deep mining, categorization and detection of significant radiology image findings using large-scale clinical lesion annotations,” arXiv preprint arXiv:1710.01766, 2017.
  5. D. C. Castro, I. Walker, and B. Glocker, “Causality matters in medical imaging,” Nature Communications, vol. 11, no. 1, pp. 1–10, 2020.
  6. M.-H. Laves, S. Ihler, J. F. Fast, L. A. Kahrs, and T. Ortmaier, “Well-calibrated regression uncertainty in medical imaging with deep learning,” in Medical Imaging with Deep Learning.   PMLR, 2020, pp. 393–412.
  7. K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer learning,” Journal of Big Data, vol. 3, no. 1, pp. 1–40, 2016.
  8. M. Raghu, C. Zhang, J. Kleinberg, and S. Bengio, “Transfusion: Understanding transfer learning for medical imaging,” Advances in Neural Information Processing Systems, vol. 32, 2019.
  9. J. Wang, X. Du, K. Farrahi, and M. Niranjan, “Deep cascade learning for optimal medical image feature representation,” Machine Learning for Healthcare (MLHC), 2022.
  10. C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” Journal of Big Data, vol. 6, no. 1, pp. 1–48, 2019.
  11. Y. Wang, Q. Yao, J. T. Kwok, and L. M. Ni, “Generalizing from a few examples: A survey on few-shot learning,” ACM Computing Surveys (CSUR), vol. 53, no. 3, pp. 1–34, 2020.
  12. A. Nichol and J. Schulman, “Reptile: a scalable metalearning algorithm,” arXiv preprint arXiv:1803.02999, vol. 2, no. 3, p. 4, 2018.
  13. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International Conference on Machine Learning.   PMLR, 2017, pp. 1126–1135.
  14. J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” Advances in Neural Information Processing Systems, vol. 30, 2017.
  15. O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra et al., “Matching networks for one shot learning,” Advances in Neural Information Processing Systems, vol. 29, 2016.
  16. J. Xu, W. An, L. Zhang, and D. Zhang, “Sparse, collaborative, or nonnegative representation: which helps pattern classification?” Pattern Recognition, vol. 88, pp. 679–688, 2019.
  17. D. Papailiopoulos, A. Dimakis, and S. Korokythakis, “Sparse pca through low-rank approximations,” in International Conference on Machine Learning.   PMLR, 2013, pp. 747–755.
  18. M. Raghu, J. Gilmer, J. Yosinski, and J. Sohl-Dickstein, “Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability,” Advances in Neural Information Processing Systems, vol. 30, 2017.
  19. C. Simon, P. Koniusz, R. Nock, and M. Harandi, “Adaptive subspaces for few-shot learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 4136–4145.
  20. T.-H. Chan, K. Jia, S. Gao, J. Lu, Z. Zeng, and Y. Ma, “Pcanet: A simple deep learning baseline for image classification?” IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5017–5032, 2015.
  21. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, 1997.
  22. D. D. Lee and H. S. Seung, “Learning the parts of objects by non-negative matrix factorization,” Nature, vol. 401, no. 6755, pp. 788–791, 1999.
  23. M. Babaee, S. Tsoukalas, M. Babaee, G. Rigoll, and M. Datcu, “Discriminative nonnegative matrix factorization for dimensionality reduction,” Neurocomputing, vol. 173, pp. 212–223, 2016.
  24. X. Cai and F. Sun, “Supervised and constrained nonnegative matrix factorization with sparseness for image representation,” Wireless Personal Communications, vol. 102, no. 4, pp. 3055–3066, 2018.
  25. D. Lee and H. S. Seung, “Algorithms for non-negative matrix factorization,” in Advances in Neural Information Processing Systems, T. Leen, T. Dietterich, and V. Tresp, Eds., vol. 13.   MIT Press, 2001. [Online]. Available: https://proceedings.neurips.cc/paper/2000/file/f9d1152547c0bde01830b7e8bd60024c-Paper.pdf
  26. A. Dong, Z. Li, and Q. Zheng, “Transferred subspace learning based on non-negative matrix factorization for eeg signal classification,” Frontiers in Neuroscience, vol. 15, 2021.
  27. J. Leuschner, M. Schmidt, P. Fernsel, D. Lachmund, T. Boskamp, and P. Maass, “Supervised non-negative matrix factorization methods for maldi imaging applications,” Bioinformatics, vol. 35, no. 11, pp. 1940–1947, 2019.
  28. Z. Chen, S. Jin, R. Liu, and J. Zhang, “A deep non-negative matrix factorization model for big data representation learning,” Frontiers in Neurorobotics, p. 93, 2021.
  29. H. Liu, Z. Wu, X. Li, D. Cai, and T. S. Huang, “Constrained nonnegative matrix factorization for image representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1299–1311, 2011.
  30. D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor, “Canonical correlation analysis: An overview with application to learning methods,” Neural Computation, vol. 16, no. 12, pp. 2639–2664, 2004.
  31. A. Janowczyk and A. Madabhushi, “Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases,” Journal of Pathology Informatics, vol. 7, 2016.
  32. J. Cheng, “brain tumor dataset,” Apr 2017. [Online]. Available: https://figshare.com/articles/dataset/brain_tumor_dataset/1512427/5
  33. J. Zhao, Y. Zhang, X. He, and P. Xie, “Covid-ct-dataset: a ct scan dataset about covid-19,” arXiv preprint arXiv:2003.13865, 2020.
  34. R. Liu, X. Wang, Q. Wu, L. Dai, X. Fang, T. Yan, J. Son, S. Tang, J. Li, Z. Gao et al., “Deepdrid: Diabetic retinopathy—grading and image quality estimation challenge,” Patterns, p. 100512, 2022.
  35. A. Acevedo, A. Merino, S. Alférez, Á. Molina, L. Boldú, and J. Rodellar, “A dataset of microscopic peripheral blood cell images for development of automatic recognition systems,” Data in Brief, ISSN: 23523409, Vol. 30,(2020), 2020.
  36. W. Al-Dhabyani, M. Gomaa, H. Khaled, and A. Fahmy, “Dataset of breast ultrasound images,” Data in Brief, vol. 28, p. 104863, 2020.
  37. P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific Data, vol. 5, no. 1, pp. 1–9, 2018.
  38. N. C. Codella, D. Gutman, M. E. Celebi, B. Helba, M. A. Marchetti, S. W. Dusza, A. Kalloo, K. Liopyris, N. Mishra, H. Kittler et al., “Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic),” in 2018 IEEE 15th International symposium on Biomedical Imaging (ISBI 2018).   IEEE, 2018, pp. 168–172.
  39. D. S. Kermany, M. Goldbaum, W. Cai, C. C. Valentim, H. Liang, S. L. Baxter, A. McKeown, G. Yang, X. Wu, F. Yan et al., “Identifying medical diagnoses and treatable diseases by image-based deep learning,” Cell, vol. 172, no. 5, pp. 1122–1131, 2018.
  40. P. Bilic, P. F. Christ, E. Vorontsov, G. Chlebus, H. Chen, Q. Dou, C. W. Fu, X. Han, P.-A. Heng, J. Hesser et al., “The liver tumor segmentation benchmark (LiTS),” arXiv preprint arXiv:1901.04056, 2019.
  41. J. N. Kather, J. Krisam, P. Charoentong, T. Luedde, E. Herpel, C.-A. Weis, T. Gaiser, A. Marx, N. A. Valous, D. Ferber et al., “Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study,” PLoS Medicine, vol. 16, no. 1, p. e1002730, 2019.
  42. A. Woloshuk, S. Khochare, A. F. Almulhim, A. T. McNutt, D. Dean, D. Barwinska, M. J. Ferkowicz, M. T. Eadon, K. J. Kelly, K. W. Dunn et al., “In situ classification of cell types in human kidney tissue using 3d nuclear staining,” Cytometry Part A, vol. 99, no. 7, pp. 707–721, 2021.
  43. V. Ljosa, K. L. Sokolnicki, and A. E. Carpenter, “Annotated high-throughput microscopy image sets for validation.” Nature Methods, vol. 9, no. 7, pp. 637–637, 2012.
  44. A. Cruz-Roa, A. Basavanhally, F. González, H. Gilmore, M. Feldman, S. Ganesan, N. Shih, J. Tomaszewski, and A. Madabhushi, “Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks,” in Medical Imaging 2014: Digital Pathology, vol. 9041.   SPIE, 2014, p. 904103.
  45. J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification,” arXiv preprint arXiv:2110.14795, 2021.
  46. Pytorch, “Pytorch, forward and backward function hooks—pytorch documentation.” [Online]. Available: https://pytorch.org/tutorials/beginner/former_torchies/nnft_tutorial.html#forward-and-backward-function-hooks
  47. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, “Learning deep features for discriminative localization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2921–2929.
  48. K. Alomar, H. I. Aysel, and X. Cai, “Data augmentation in classification and segmentation: a survey and new strategies,” Journal of Imaging, vol. 9, 2023. [Online]. Available: https://doi.org/10.3390/jimaging9020046
  49. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “The omniglot challenge: a 3-year progress report,” Current Opinion in Behavioral Sciences, vol. 29, pp. 97–104, 2019.
  50. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now