Investigating the Robustness of Vision Transformers against Label Noise in Medical Image Classification (2402.16734v1)
Abstract: Label noise in medical image classification datasets significantly hampers the training of supervised deep learning methods, undermining their generalizability. The test performance of a model tends to decrease as the label noise rate increases. Over recent years, several methods have been proposed to mitigate the impact of label noise in medical image classification and enhance the robustness of the model. Predominantly, these works have employed CNN-based architectures as the backbone of their classifiers for feature extraction. However, in recent years, Vision Transformer (ViT)-based backbones have replaced CNNs, demonstrating improved performance and a greater ability to learn more generalizable features, especially when the dataset is large. Nevertheless, no prior work has rigorously investigated how transformer-based backbones handle the impact of label noise in medical image classification. In this paper, we investigate the architectural robustness of ViT against label noise and compare it to that of CNNs. We use two medical image classification datasets -- COVID-DU-Ex, and NCT-CRC-HE-100K -- both corrupted by injecting label noise at various rates. Additionally, we show that pretraining is crucial for ensuring ViT's improved robustness against label noise in supervised training.
- Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928, 2020.
- Training a neural network based on unreliable human annotation of medical images. In 2018 IEEE 15th International symposium on biomedical imaging (ISBI 2018), pages 39–42. IEEE, 2018.
- An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
- Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018.
- A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence, 45(1):87–110, 2022.
- Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022.
- Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 590–597, 2019.
- Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study. PLoS medicine, 16(1):e1002730, 2019.
- Improving medical image classification in noisy labels using only self-supervised pretraining. In MICCAI Workshop on Data Engineering in Medical Imaging, pages 78–90. Springer, 2023.
- Investigating the impact of class-dependent label noise in medical image classification. In Medical Imaging 2023: Image Processing, volume 12464, pages 728–733. SPIE, 2023.
- How does heterogeneous label noise impact generalization in neural nets? In Advances in Visual Computing: 16th International Symposium, ISVC 2021, Virtual Event, October 4-6, 2021, Proceedings, Part II, pages 229–241. Springer, 2021.
- Pancreatic cancer detection in whole slide images using noisy label annotations. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22, pages 541–549. Springer, 2019.
- Robust inference via generative classifiers for handling noisy labels. In International conference on machine learning, pages 3763–3772. PMLR, 2019.
- Dividemix: Learning with noisy labels as semi-supervised learning. In International Conference on Learning Representations, 2020.
- Selective-supervised contrastive learning with noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 316–325, 2022.
- Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
- A survey of crowdsourcing in medical image analysis. Human Computation, 7:1–26, 2020.
- Robustness to label noise depends on the shape of the noise distribution. Advances in Neural Information Processing Systems, 35:35645–35656, 2022.
- Interpreting chest x-rays via cnns that exploit hierarchical disease dependencies and uncertainty labels. Neurocomputing, 437:186–194, 2021.
- Labelling instructions matter in biomedical image analysis. Nature Machine Intelligence, 5(3):273–283, 2023.
- Rochester Institute of Technology. Research computing services, 2022.
- Transformers in medical imaging: A survey. Medical Image Analysis, page 102802, 2023.
- Covid-19 infection localization and severity grading from chest x-ray images. Computers in biology and medicine, 139:105002, 2021.
- Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663, 2022.
- Robust learning at noisy labeled medical images: Applied to skin lesion classification. In 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019), pages 1280–1283. IEEE, 2019.
- Robust medical image classification from noisy labeled data with global and local representation guided co-training. IEEE Transactions on Medical Imaging, 41(6):1371–1382, 2022.
- Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107–115, 2021.
- Combating medical noisy labels by disentangled distribution learning and consistency regularization. Future Generation Computer Systems, 141:567–576, 2023.