Focused Active Learning for Histopathological Image Classification (2404.04663v1)
Abstract: Active Learning (AL) has the potential to solve a major problem of digital pathology: the efficient acquisition of labeled data for machine learning algorithms. However, existing AL methods often struggle in realistic settings with artifacts, ambiguities, and class imbalances, as commonly seen in the medical field. The lack of precise uncertainty estimations leads to the acquisition of images with a low informative value. To address these challenges, we propose Focused Active Learning (FocAL), which combines a Bayesian Neural Network with Out-of-Distribution detection to estimate different uncertainties for the acquisition function. Specifically, the weighted epistemic uncertainty accounts for the class imbalance, aleatoric uncertainty for ambiguous images, and an OoD score for artifacts. We perform extensive experiments to validate our method on MNIST and the real-world Panda dataset for the classification of prostate cancer. The results confirm that other AL methods are 'distracted' by ambiguities and artifacts which harm the performance. FocAL effectively focuses on the most informative images, avoiding ambiguities and artifacts during acquisition. For both experiments, FocAL outperforms existing AL approaches, reaching a Cohen's kappa of 0.764 with only 0.69% of the labeled Panda data.
- Z. Zhang, P. Chen, M. McGough, F. Xing, C. Wang, M. Bui, Y. Xie, M. Sapkota, L. Cui, J. Dhillon, N. Ahmad, F. K. Khalil, S. I. Dickinson, X. Shi, F. Liu, H. Su, J. Cai, and L. Yang, “Pathologist-level interpretable whole-slide cancer diagnosis with deep learning,” Nature Machine Intelligence, vol. 1, no. 5, pp. 236–245, 2019.
- A. Hekler, J. S. Utikal, A. H. Enk, W. Solass, M. Schmitt, J. Klode, D. Schadendorf, W. Sondermann, C. Franklin, F. Bestvater, M. J. Flaig, D. Krahl, C. von Kalle, S. Fröhling, and T. J. Brinker, “Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images,” European Journal of Cancer, vol. 118, pp. 91–96, 2019.
- B. Ehteshami Bejnordi, M. Veta, P. Johannes van Diest, B. van Ginneken, N. Karssemeijer, G. Litjens, J. A. W. M. van der Laak, and a. t. C. Consortium, “Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer,” JAMA, vol. 318, no. 22, pp. 2199–2210, 2017.
- N. Dimitriou, O. Arandjelović, and P. D. Caie, “Deep learning for whole slide image analysis: An overview,” Frontiers in Medicine, vol. 6, p. 264, 2019.
- N. Shvetsov, M. Grønnesby, E. Pedersen, K. Møllersen, L.-T. R. Busund, R. Schwienbacher, L. A. Bongo, and T. K. Kilvaer, “A pragmatic machine learning approach to quantify tumor-infiltrating lymphocytes in whole slide images,” Cancers, vol. 14, no. 12, p. 2974, 2022.
- A. Schmidt, J. Silva-Rodriguez, R. Molina, and V. Naranjo, “Efficient cancer classification by coupling semi supervised and multiple instance learning,” IEEE Access, vol. 10, pp. 9763–9773, 2022.
- W. Bulten, K. Kartasalo, P.-H. C. Chen, P. Ström, H. Pinckaers, K. Nagpal, Y. Cai, D. F. Steiner, H. van Boven, R. Vink, C. Hulsbergen-van de Kaa, J. van der Laak, M. B. Amin, A. J. Evans, T. van der Kwast, R. Allan, P. A. Humphrey, H. Grönberg, H. Samaratunga, B. Delahunt, T. Tsuzuki, T. Häkkinen, L. Egevad, M. Demkin, S. Dane, F. Tan, M. Valkonen, G. S. Corrado, L. Peng, C. H. Mermel, P. Ruusuvuori, G. Litjens, M. Eklund, the PANDA challenge consortium, A. Brilhante, A. Çakır, X. Farré, K. Geronatsiou, V. Molinié, G. Pereira, P. Roy, G. Saile, P. G. O. Salles, E. Schaafsma, J. Tschui, J. Billoch-Lima, E. M. Pereira, M. Zhou, S. He, S. Song, Q. Sun, H. Yoshihara, T. Yamaguchi, K. Ono, T. Shen, J. Ji, A. Roussel, K. Zhou, T. Chai, N. Weng, D. Grechka, M. V. Shugaev, R. Kiminya, V. Kovalev, D. Voynov, V. Malyshev, E. Lapo, M. Campos, N. Ota, S. Yamaoka, Y. Fujimoto, K. Yoshioka, J. Juvonen, M. Tukiainen, A. Karlsson, R. Guo, C.-L. Hsieh, I. Zubarev, H. S. T. Bukhar, W. Li, J. Li, W. Speier, C. Arnold, K. Kim, B. Bae, Y. W. Kim, H.-S. Lee, and J. Park, “Artificial intelligence for diagnosis and gleason grading of prostate cancer: the PANDA challenge,” Nature Medicine, vol. 28, no. 1, pp. 154–163, 2022.
- S. Otálora, N. Marini, H. Müller, and M. Atzori, “Semi-weakly supervised learning for prostate cancer image classification with teacher-student deep convolutional networks,” vol. 12446 LNCS, pp. 193–203, 2020.
- J. Li, W. Speier, K. C. Ho, K. V. Sarma, A. Gertych, B. S. Knudsen, and C. W. Arnold, “An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies,” Comput. Medical Imaging Graph., vol. 69, pp. 125–133, 2018.
- N. Marini, S. Otálora, H. Müller, and M. Atzori, “Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations: An experiment on prostate histopathology image classification,” Medical Image Analysis, vol. 73, p. 102165, 2021.
- M. Y. Lu, R. J. Chen, and F. Mahmood, “Semi-supervised breast cancer histology classification using deep multiple instance learning and contrast predictive coding (conference presentation),” in Medical Imaging 2020: Digital Pathology, 2020, p. 18.
- G. Campanella, M. G. Hanna, L. Geneslaw, A. Miraflor, V. Werneck Krauss Silva, K. J. Busam, E. Brogi, V. E. Reuter, D. S. Klimstra, and T. J. Fuchs, “Clinical-grade computational pathology using weakly supervised deep learning on whole slide images,” Nature Medicine, vol. 25, no. 8, pp. 1301–1309, 2019.
- P. Chikontwe, M. Kim, S. J. Nam, H. Go, and S. H. Park, “Multiple instance learning with center embeddings for histopathology classification,” in Medical Image Computing and Computer Assisted Intervention – MICCAI, 2020, pp. 519–528.
- B. Li, Y. Li, and K. W. Eliceiri, “Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning,” in Conference on Computer Vision and Pattern Recognition - CVPR, 2021, pp. 14 313–14 323.
- A. J. Joshi, F. Porikli, and N. Papanikolopoulos, “Multi-class active learning for image classification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 2372–2379.
- X. Li and Y. Guo, “Adaptive active learning for image classification,” in IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 859–866.
- X. Zhu, J. Lafferty, and Z. Ghahramani, “Combining active learning and semi-supervised learning using gaussian fields and harmonic functions,” in International Conference on Machine Learning - ICML, 2003, pp. 58–65.
- Y. Gal, R. Islam, and Z. Ghahramani, “Deep bayesian active learning with image data,” International Conference on Machine Learning - ICML, pp. 1183–1192, 2017.
- L. Raczkowski, M. Możejko, J. Zambonelli, and E. Szczurek, “ARA: accurate, reliable and active histopathological image classification framework with bayesian deep learning,” Scientific Reports, vol. 9, no. 1, p. 14347, 2019.
- J. Carse and S. McKenna, “Active learning for patch-based digital pathology using convolutional neural networks to reduce annotation costs,” in Digital Pathology, 2019, vol. 11435, pp. 20–27.
- A. L. Meirelles, T. Kurc, J. Saltz, and G. Teodoro, “Effective active learning in digital pathology: A case study in tumor infiltrating lymphocytes,” Computer Methods and Programs in Biomedicine, vol. 220, p. 106828, 2022.
- N. Houlsby, F. Huszar, Z. Ghahramani, and M. Lengyel, “Bayesian active learning for classification and preference learning,” CoRR, vol. abs/1112.5745, 2011. [Online]. Available: http://arxiv.org/abs/1112.5745
- C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, 1948.
- V. B. Alex Kendall and R. Cipolla, “Bayesian SegNet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding,” in British Machine Vision Conference - BMVC, 2017, pp. 57.1–57.12.
- S. Lee, M. Amgad, P. Mobadersany, M. McCormick, B. P. Pollack, H. Elfandy, H. Hussein, D. A. Gutman, and L. A. Cooper, “Interactive classification of whole-slide imaging data for cancer researchers,” Cancer Research, vol. 81, no. 4, pp. 1171–1177, 2021.
- R. Marée, L. Rollus, B. Stévens, R. Hoyoux, G. Louppe, R. Vandaele, J.-M. Begon, P. Kainz, P. Geurts, and L. Wehenkel, “Collaborative analysis of multi-gigapixel imaging data using cytomine,” Bioinformatics, vol. 32, no. 9, pp. 1395–1401, 2016.
- Y. Kwon, J.-H. Won, B. J. Kim, and M. C. Paik, “Uncertainty quantification using bayesian neural networks in classification: Application to biomedical image segmentation,” Computational Statistics & Data Analysis, vol. 142, p. 106816, 2020.
- A. Der Kiureghian and O. Ditlevsen, “Aleatory or epistemic? does it matter?” Structural safety, vol. 31, no. 2, pp. 105–112, 2009, publisher: Elsevier.
- T. Xiao, A. Gomez, and Y. Gal, “Wat heb je gezegd? detecting out-of-distribution translations with variational transformers,” 2019.
- J. Mukhoti, A. Kirsch, J. van Amersfoort, P. H. S. Torr, and Y. Gal, “Deep deterministic uncertainty: A simple baseline,” 2021. [Online]. Available: 10.48550/ARXIV.2102.11582
- A. T. Nguyen, F. Lu, G. L. Munoz, E. Raff, C. Nicholas, and J. Holt, “Out of distribution data detection using dropout bayesian neural networks,” 2022. [Online]. Available: 10.48550/ARXIV.2202.08985
- B.-B. Gao, C. Xing, C.-W. Xie, J. Wu, and X. Geng, “Deep label distribution learning with label ambiguity,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2825–2838, 2017.
- M. Bernhardt, D. Coelho de Castro, R. Tanno, A. Schwaighofer, K. Tezcan, M. Monteiro, S. Bannur, M. Lungren, A. Nori, B. Glocker, J. Alvarez-Valle, and O. Oktay, “Active label cleaning for improved dataset quality under resource constraints,” Nature Communications, vol. 13, 2022.
- A. Kendall and Y. Gal, “What uncertainties do we need in bayesian deep learning for computer vision?” in Neural Information Processing Systems - NeurIPS, vol. 30, 2017.
- Y. Sun, Y. Ming, X. Zhu, and Y. Li, “Out-of-distribution detection with deep nearest neighbors,” in International Conference on Machine Learning - ICML, 2022.
- N. Kanwal, F. Pérez-Bueno, A. Schmidt, R. Molina, and K. Engan, “The devil is in the details: Whole slide image acquisition and processing for artifacts detection, color variation, and data augmentation: A review,” IEEE Access, vol. 10, 2022.
- T. Johnson, I. Kwok, and R. T. Ng, “Fast computation of 2-dimensional depth contours.” in International Conference on Knowledge Discovery and Data Mining, 1998, pp. 224–228.
- I. Ruts and P. J. Rousseeuw, “Computing depth contours of bivariate point clouds,” Computational Statistics & Data Analysis, vol. 23, no. 1, pp. 153–168, 1996.
- E. M. Knorr and R. T. Ng, “Algorithms for mining distance-based outliers in large datasets.” in International Conference on Very Large Databases - VLDB, A. Gupta, O. Shmueli, and J. Widom, Eds., 1998, pp. 392–403.
- ——, “Finding intensional knowledge of distance-based outliers.” in International Conference on Very Large Databases - VLDB, M. P. Atkinson, M. E. Orlowska, P. Valduriez, S. B. Zdonik, and M. L. Brodie, Eds., 1999, pp. 211–222.
- D. Abati, A. Porrello, S. Calderara, and R. Cucchiara, “Latent space autoregression for novelty detection,” in Conference on Computer Vision and Pattern Recognition - CVPR. IEEE Computer Society, 2019, pp. 481–490.
- K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples and adversarial attacks,” in International Conference on Neural Information Processing Systems - NeurIPS, 2018, pp. 7167–7177.
- M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying density-based local outliers,” in ACM SIGMOD International Conference on Management of Data, ser. Association for Computing Machinery, 2000, pp. 93–104.
- Y. Wu, A. Schmidt, E. Hernández-Sánchez, R. Molina, and A. K. Katsaggelos, “Combining attention-based multiple instance learning and gaussian processes for CT hemorrhage detection,” in Medical Image Computing and Computer Assisted Intervention – MICCAI, 2021, vol. 12902, pp. 582–591.
- A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun, “Dermatologist-level classification of skin cancer with deep neural networks,” Nature, vol. 542, no. 7639, pp. 115–118, 2017.
- V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, R. Kim, R. Raman, P. C. Nelson, J. L. Mega, and D. R. Webster, “Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs,” Journal of the American Medical Association -AMA, vol. 316, no. 22, pp. 2402–2410, 2016-12.
- J. Zeng, A. Lesnikowski, and J. M. Alvarez, “The relevance of bayesian layer positioning to model uncertainty in deep bayesian active learning,” Neural Information Processing Systems - NeurIPS, 2018.
- D. P. Kingma, T. Salimans, and M. Welling, “Variational dropout and the local reparameterization trick,” in International Conference on Neural Information Processing Systems - NIPS, 2015, pp. 2575–2583.
- V.-L. Nguyen, S. Destercke, and E. Hüllermeier, “Epistemic uncertainty sampling,” in International Conference on Discovery Science, P. Kralj Novak, T. Šmuc, and S. Džeroski, Eds., 2019, pp. 72–86.
- L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012.
- A. Oliver, A. Odena, C. Raffel, E. D. Cubuk, and I. J. Goodfellow, “Realistic evaluation of deep semi-supervised learning algorithms,” in International Conference on Neural Information Processing Systems - NIPS. Curran Associates Inc., 2018, pp. 3239–3250.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2015.
- M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in International Conference on Machine Learning - ICML, vol. 97, 2019, pp. 6105–6114.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Conference on Computer Vision and Pattern Recognition - CVPR, 2016, pp. 770–778.