Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep learning with noisy labels in medical prediction problems: a scoping review (2403.13111v1)

Published 19 Mar 2024 in cs.LG and cs.AI

Abstract: Objectives: Medical research faces substantial challenges from noisy labels attributed to factors like inter-expert variability and machine-extracted labels. Despite this, the adoption of label noise management remains limited, and label noise is largely ignored. To this end, there is a critical need to conduct a scoping review focusing on the problem space. This scoping review aims to comprehensively review label noise management in deep learning-based medical prediction problems, which includes label noise detection, label noise handling, and evaluation. Research involving label uncertainty is also included. Methods: Our scoping review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched 4 databases, including PubMed, IEEE Xplore, Google Scholar, and Semantic Scholar. Our search terms include "noisy label AND medical / healthcare / clinical", "un-certainty AND medical / healthcare / clinical", and "noise AND medical / healthcare / clinical". Results: A total of 60 papers met inclusion criteria between 2016 and 2023. A series of practical questions in medical research are investigated. These include the sources of label noise, the impact of label noise, the detection of label noise, label noise handling techniques, and their evaluation. Categorization of both label noise detection methods and handling techniques are provided. Discussion: From a methodological perspective, we observe that the medical community has been up to date with the broader deep-learning community, given that most techniques have been evaluated on medical data. We recommend considering label noise as a standard element in medical research, even if it is not dedicated to handling noisy labels. Initial experiments can start with easy-to-implement methods, such as noise-robust loss functions, weighting, and curriculum learning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity. Journal of American Association for Pediatric Ophthalmology and Strabismus. 2008;12(4):352-6.
  2. A comprehensive introduction to label noise. In: ESANN. Citeseer; 2014. .
  3. Learning From Noisy Labels With Deep Neural Networks: A Survey. IEEE Trans Neural Netw Learn Syst. 2023 Nov;34(11):8135-53.
  4. Algan G, Ulusoy I. Image classification with deep learning in the presence of noisy labels: A survey. Knowledge-Based Systems. 2021;215:106771.
  5. Review–a survey of learning from noisy labels. ECS Sensors Plus. 2022;1(2):021401.
  6. Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis. Medical image analysis. 2020;65:101759.
  7. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Annals of internal medicine. 2018;169(7):467-73.
  8. Deep and structured robust information theoretic learning for image analysis. IEEE Transactions on Image Processing. 2016;25(9):4209-21.
  9. Training a neural network based on unreliable human annotation of medical images. In: 2018 IEEE 15th International symposium on biomedical imaging (ISBI 2018). IEEE; 2018. p. 39-42.
  10. Robust learning at noisy labeled medical images: Applied to skin lesion classification. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE; 2019. p. 1280-3.
  11. Improving medical images classification with label noise using dual-uncertainty estimation. IEEE transactions on medical imaging. 2022;41(6):1533-46.
  12. Robust medical image classification from noisy labeled data with global and local representation guided co-training. IEEE Transactions on Medical Imaging. 2022;41(6):1371-82.
  13. Label-noise-tolerant medical image classification via self-attention and self-supervised learning. arXiv preprint arXiv:230609718. 2023.
  14. Deep supervised learning using self-adaptive auxiliary loss for COVID-19 diagnosis from imbalanced CT images. Neurocomputing. 2021;458:232-45.
  15. Fully automated plaque characterization in intravascular OCT images using hybrid convolutional and lumen morphology features. Scientific reports. 2020;10(1):2596.
  16. Accurate deep learning model using semi-supervised learning and Noisy Student for cervical cancer screening in low magnification images. Plos one. 2023;18(5):e0285996.
  17. Weakly supervised classification of aortic valve malformations using unlabeled cardiac MRI sequences. Nature communications. 2019;10(1):3111.
  18. Deep learning from multiple experts improves identification of amyloid neuropathologies. Acta neuropathologica communications. 2022;10(1):66.
  19. A loss-based patch label denoising method for improving whole-slide image analysis using a convolutional neural network. Scientific reports. 2022;12(1):1392.
  20. COVID-19 chest X-ray image classification in the presence of noisy labels. Displays. 2023;77:102370.
  21. Learning from crowds in digital pathology using scalable variational Gaussian processes. Scientific reports. 2021;11(1):11612.
  22. Learning to detect brain lesions from noisy annotations. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE; 2020. p. 1910-4.
  23. Advancing Brain Metastases Detection in T1-Weighted Contrast-Enhanced 3D MRI Using Noisy Student-Based Training. Diagnostics. 2022;12(8):2023.
  24. Learning-to-augment strategy using noisy and denoised data: Improving generalizability of deep CNN for the detection of COVID-19 in X-ray images. Computers in Biology and Medicine. 2021;136:104704.
  25. Ros-kd: A robust stochastic knowledge distillation approach for noisy medical imaging. In: 2022 IEEE International Conference on Data Mining (ICDM). IEEE; 2022. p. 981-6.
  26. Semi-supervised classification of noisy, gigapixel histology images. In: 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). IEEE; 2020. p. 563-8.
  27. Generalized zero-shot chest x-ray diagnosis through trait-guided multi-view semantic embedding with self-training. IEEE Transactions on Medical Imaging. 2021;40(10):2642-55.
  28. Pathal: An active learning framework for histopathology image analysis. IEEE Transactions on Medical Imaging. 2021;41(5):1176-87.
  29. Reliable label-efficient learning for biomedical image recognition. IEEE Transactions on Biomedical Engineering. 2018;66(9):2423-32.
  30. REUR: a unified deep framework for signet ring cell detection in low-resolution pathological images. Computers in Biology and Medicine. 2021;136:104711.
  31. Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment. Medical Image Analysis. 2021;72:102087.
  32. Deep learning from small amount of medical data with noisy labels: A meta-learning approach. arXiv preprint arXiv:201006939. 2020.
  33. Quantifying and leveraging classification uncertainty for chest radiograph assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. Springer; 2019. p. 676-84.
  34. Interpreting chest X-rays via CNNs that exploit hierarchical disease dependencies and uncertainty labels. Neurocomputing. 2021;437:186-94.
  35. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33; 2019. p. 590-7.
  36. Learning Robust Classifier for Imbalanced Medical Image Dataset with Noisy Labels by Minimizing Invariant Risk. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2023. p. 306-16.
  37. Adaptive Cross Entropy for ultrasmall object detection in Computed Tomography with noisy labels. Computers in Biology and Medicine. 2022;147:105763.
  38. Automatic diagnosis and grading of Prostate Cancer with weakly supervised learning on whole slide images. Computers in Biology and Medicine. 2023;152:106340.
  39. Labeling confidence for uncertainty-aware histology image classification. Computerized Medical Imaging and Graphics. 2023;107:102231.
  40. Handling label noise through model confidence and uncertainty: application to chest radiograph classification. In: Medical Imaging 2019: Computer-Aided Diagnosis. vol. 10950. SPIE; 2019. p. 289-96.
  41. Influence Based Re-Weighing for Labeling Noise in Medical Imaging. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI). IEEE; 2022. p. 1-5.
  42. Curriculum learning for improved femur fracture classification: Scheduling data with prior knowledge and uncertainty. Medical Image Analysis. 2022;75:102273.
  43. Co-correcting: noise-tolerant medical image classification via mutual label correction. IEEE Transactions on Medical Imaging. 2021;40(12):3580-92.
  44. A fundus image classification framework for learning with noisy labels. Computerized Medical Imaging and Graphics. 2023;108:102278.
  45. Correcting Pseudo Labels with Label Distribution for Unsupervised Domain Adaptive Vulnerable Plaque Detection. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE; 2021. p. 3225-8.
  46. Clinical knowledge embedded method based on multi-task learning for thyroid nodule classification with ultrasound images. Physics in Medicine & Biology. 2023;68(4):045018.
  47. Bayesian statistics-guided label refurbishment mechanism: Mitigating label noise in medical image classification. Medical Physics. 2022;49(9):5899-913.
  48. ReFixMatch-LS: reusing pseudo-labels for semi-supervised skin lesion classification. Medical & Biological Engineering & Computing. 2023;61(5):1033-45.
  49. Robust co-teaching learning with consistency-based noisy label correction for medical image classification. International Journal of Computer Assisted Radiology and Surgery. 2023;18(4):675-83.
  50. Training deep neural networks with noisy clinical labels: toward accurate detection of prostate cancer in US data. International Journal of Computer Assisted Radiology and Surgery. 2022;17(9):1697-705.
  51. Combating Medical Label Noise via Robust Semi-supervised Contrastive Learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer; 2023. p. 562-72.
  52. Bomd: bag of multi-label descriptors for noisy chest x-ray classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2023. p. 21284-95.
  53. Alternating loss correction for preterm-birth prediction from ehr data with noisy labels. arXiv preprint arXiv:181109782. 2018.
  54. Addressing Label Noise for Electronic Health Records: Insights from Computer Vision for Tabular Data. medRxiv. 2023:2023-10.
  55. Automated and flexible identification of complex disease: building a model for systemic lupus erythematosus using noisy labeling. Journal of the American Medical Informatics Association. 2019;26(1):61-5.
  56. Dhrangadhariya A, Müller H. Not so weak PICO: leveraging weak supervision for participants, interventions, and outcomes recognition for systematic review automation. JAMIA open. 2023;6(1):ooac107.
  57. Semi-supervised noisy label learning for Chinese clinical named entity recognition. Data Intelligence. 2021;3(3):389-401.
  58. Label noise and self-learning label correction in cardiac abnormalities classification. Physiological measurement. 2022;43(9):094001.
  59. Stochastic co-teaching for training neural networks with unknown levels of label noise. Scientific reports. 2023;13(1):16875.
  60. Automatic diagnosis of multiple cardiac diseases from PCG signals using convolutional neural network. Computer Methods and Programs in Biomedicine. 2020;197:105750.
  61. Two will do: CNN with asymmetric loss, self-learning label correction, and hand-crafted features for imbalanced multi-label ECG data classification. In: 2021 Computing in Cardiology (CinC). vol. 48. IEEE; 2021. p. 1-4.
  62. Learning From Alarms: A Robust Learning Approach for Accurate Photoplethysmography-Based Atrial Fibrillation Detection using Eight Million Samples Labeled with Imprecise Arrhythmia Alarms. arXiv preprint arXiv:221103333. 2022.
  63. Semi-Supervised Calibration of Noisy Event Risk (SCANER) with Electronic Health Records. Journal of Biomedical Informatics. 2023:104425.
  64. OCRFinder: a noise-tolerance machine learning method for accurately estimating open chromatin regions. Frontiers in Genetics. 2023;14:1184744.
  65. Tjandra D, Wiens J. Leveraging an Alignment Set in Tackling Instance-Dependent Label Noise. In: Conference on Health, Inference, and Learning. PMLR; 2023. p. 477-97.
  66. Improving Medical Predictions with Label Noise Tolerant Classification. In: 2022 4th International Conference on Advances in Computing, Communication Control and Networking (ICAC3N). IEEE; 2022. p. 765-9.
  67. Hybrid label noise correction algorithm for medical auxiliary diagnosis. In: 2020 IEEE 18th International Conference on Industrial Informatics (INDIN). vol. 1. IEEE; 2020. p. 567-72.
  68. Brady AP. Error and discrepancy in radiology: inevitable or avoidable? Insights into imaging. 2017;8:171-82.
  69. Automated stent coverage analysis in intravascular OCT (IVOCT) image volumes using a support vector machine and mesh growing. Biomedical Optics Express. 2019;10(6):2809-28.
  70. Effects of label noise on deep learning-based skin cancer classification. Frontiers in Medicine. 2020;7:177.
  71. Plus disease in rop: why do experts disagree, and how can we improve diagnosis? Journal of American Association for Pediatric Ophthalmology and Strabismus {{\{{JAAPOS}}\}}. 2017;21(4):e5-6.
  72. Inference of chronic obstructive pulmonary disease with deep learning on raw spirograms identifies new genetic loci and improves risk models. Nature Genetics. 2023:1-9.
  73. Impact of label noise on the learning based models for a binary classification of physiological signal. Sensors. 2022;22(19):7166.
  74. Class noise and supervised learning in medical domains: The effect of feature extraction. In: 19th IEEE symposium on computer-based medical systems (CBMS’06). IEEE; 2006. p. 708-13.
  75. Detection of oedema on optical coherence tomography images using deep learning model trained on noisy clinical data. Acta Ophthalmologica. 2022;100(1):103-10.
  76. Investigating the impact of class-dependent label noise in medical image classification. In: Medical Imaging 2023: Image Processing. vol. 12464. SPIE; 2023. p. 728-33.
  77. Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis. Physics in Medicine & Biology. 2020;65(10):105002.
  78. Impact of Noisy Labels on Dental Deep Learning—Calculus Detection on Bitewing Radiographs. Journal of Clinical Medicine. 2023;12(9):3058.
  79. Assessment of the robustness of convolutional neural networks in labeling noise by using chest X-ray images from multiple centers. JMIR medical informatics. 2020;8(8):e18089.
  80. The path toward equal performance in medical machine learning. Patterns. 2023;4(7).
  81. Liu T, Tao D. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence. 2015;38(3):447-61.
  82. Making deep neural networks robust to label noise: A loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 1944-52.
  83. Goldberger J, Ben-Reuven E. Training deep neural-networks using a noise adaptation layer. In: International conference on learning representations; 2016. .
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yishu Wei (5 papers)
  2. Yu Deng (88 papers)
  3. Cong Sun (25 papers)
  4. Mingquan Lin (19 papers)
  5. Hongmei Jiang (1 paper)
  6. Yifan Peng (147 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets