Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

HydraViT: Adaptive Multi-Branch Transformer for Multi-Label Disease Classification from Chest X-ray Images (2310.06143v1)

Published 9 Oct 2023 in eess.IV, cs.CV, and cs.LG

Abstract: Chest X-ray is an essential diagnostic tool in the identification of chest diseases given its high sensitivity to pathological abnormalities in the lungs. However, image-driven diagnosis is still challenging due to heterogeneity in size and location of pathology, as well as visual similarities and co-occurrence of separate pathology. Since disease-related regions often occupy a relatively small portion of diagnostic images, classification models based on traditional convolutional neural networks (CNNs) are adversely affected given their locality bias. While CNNs were previously augmented with attention maps or spatial masks to guide focus on potentially critical regions, learning localization guidance under heterogeneity in the spatial distribution of pathology is challenging. To improve multi-label classification performance, here we propose a novel method, HydraViT, that synergistically combines a transformer backbone with a multi-branch output module with learned weighting. The transformer backbone enhances sensitivity to long-range context in X-ray images, while using the self-attention mechanism to adaptively focus on task-critical regions. The multi-branch output module dedicates an independent branch to each disease label to attain robust learning across separate disease classes, along with an aggregated branch across labels to maintain sensitivity to co-occurrence relationships among pathology. Experiments demonstrate that, on average, HydraViT outperforms competing attention-guided methods by 1.2%, region-guided methods by 1.4%, and semantic-guided methods by 1.0% in multi-label classification performance.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. A. Saha, P. Dickinson, R. Shrimali, A. Salem, and S. Agarwal, “Is thoracic radiotherapy an absolute contraindication for treatment of lung cancer patients with interstitial lung disease? a systematic review,” Clin Oncol, vol. 34, no. 12, pp. e493–e504, 2022.
  2. T. Gupte, A. Knack, and J. D. Cramer, “Mortality from aspiration pneumonia: incidence, trends, and risk factors,” Dysphagia, vol. 37, no. 6, pp. 1493–1500, 2022.
  3. M. E. Kruk, A. D. Gage, N. T. Joseph, G. Danaei, S. García-Saisó, and J. A. Salomon, “Mortality due to low-quality health systems in the universal health coverage era: a systematic analysis of amenable deaths in 137 countries,” The Lancet, vol. 392, no. 10160, pp. 2203–2212, 2018.
  4. Q. Guan, Y. Huang, Z. Zhong, Z. Zheng, L. Zheng, and Y. Yang, “Thorax disease classification with attention guided convolutional neural network,” Pattern Recognit Let, vol. 131, pp. 38–45, 2020.
  5. H. Wang, S. Wang, Z. Qin, Y. Zhang, R. Li, and Y. Xia, “Triple attention learning for classification of 14 thoracic diseases using chest radiography,” Med Image Anal, vol. 67, p. 101846, 2021.
  6. J. Su, Z. Luo, and S. Li, “Consistent response for automated multilabel thoracic disease classification,” Concurr Comput, vol. 34, no. 23, p. e7201, 2022.
  7. H. E. Kim, A. Cosa-Linan, N. Santhanam, M. Jannesari, M. E. Maros, and T. Ganslandt, “Transfer learning for medical image classification: A literature review,” BMC Med Imaging, vol. 22, no. 1, p. 69, 2022.
  8. P. Celard, E. Iglesias, J. Sorribes-Fdez, R. Romero, A. S. Vieira, and L. Borrajo, “A survey on deep learning applied to medical images: From simple artificial neural networks to generative models,” Neural Comput App, vol. 35, no. 3, pp. 2291–2323, 2023.
  9. Q. Guan, Y. Huang, Y. Luo, P. Liu, M. Xu, and Y. Yang, “Discriminative feature learning for thorax disease classification in chest x-ray images,” IEEE Trans Image Process, vol. 30, pp. 2476–2487, 2021.
  10. H. Wang and Y. Xia, “Domain-ensemble learning with cross-domain mixup for thoracic disease classification in unseen domains,” Biomed Signal Process Control, vol. 81, p. 104488, 2023.
  11. X. Zhu, S. Pang, X. Zhang, J. Huang, L. Zhao, K. Tang et al., “Pcan: Pixel-wise classification and attention network for thoracic disease classification and weakly supervised localization,” Comput Med Imaging Graph, vol. 102, p. 102137, 2022.
  12. H.-G. Jung, W.-J. Nam, H.-W. Kim, and S.-W. Lee, “Weakly supervised thoracic disease localization via disease masks,” Neurocomput, vol. 517, pp. 34–43, 2023.
  13. B. Chen, J. Li, G. Lu, and D. Zhang, “Lesion location attention guided network for multi-label thoracic disease classification in chest x-rays,” IEEE J Biomed Health Inf, vol. 24, no. 7, pp. 2016–2027, 2019.
  14. B. Chen, Z. Zhang, J. Lin, Y. Chen, and G. Lu, “Two-stream collaborative network for multi-label chest x-ray image classification with lung segmentation,” Pattern Recognit Let, vol. 135, pp. 221–227, 2020.
  15. M. Hossain, S. Hossain, M. Zunaed, T. Hasan et al., “A novel attention mechanism using anatomical prior probability maps for thoracic disease classification from x-ray images,” arXiv:2210.02998, 2022.
  16. M. S. Lee and S. W. Han, “Duetnet: Dual encoder based transfer network for thoracic disease classification,” Pattern Recognit Let, vol. 161, pp. 143–153, 2022.
  17. F. Li, L. Zhou, Y. Wang, C. Chen, S. Yang, F. Shan et al., “Modeling long-range dependencies for weakly supervised disease classification and localization on chest x-ray,” Quant Imaging Med Surg, vol. 12, no. 6, p. 3364, 2022.
  18. H. Wang, H. Jia, L. Lu, and Y. Xia, “Thorax-net: an attention regularized deep neural network for classification of thoracic diseases on chest radiography,” IEEE J Biomed Health Inf, vol. 24, no. 2, pp. 475–485, 2019.
  19. D. Sriker, H. Greenspan, and J. Goldberger, “Class-based attention mechanism for chest radiograph multi-label categorization,” in IEEE Int Symp Biomed Imaging.   IEEE, 2022, pp. 1–5.
  20. K. Chen, X. Wang, and S. Zhang, “Thorax disease classification based on pyramidal convolution shuffle attention neural network,” IEEE Access, vol. 10, pp. 85 571–85 581, 2022.
  21. V. Teixeira, L. Braz, H. Pedrini, and Z. Dias, “Dualanet: dual lesion attention network for thoracic disease classification in chest x-rays,” in IWSSIP.   IEEE, 2020, pp. 69–74.
  22. B. Jung, L. Gu, and T. Harada, “Graph interaction for automated diagnosis of thoracic disease using x-ray images,” in SPIE Med Imaging, vol. 12032.   SPIE, 2022, pp. 135–147.
  23. H. Wang, Y.-Y. Yang, Y. Pan, P. Han, Z.-X. Li, H.-G. Huang et al., “Detecting thoracic diseases via representation learning with adaptive sampling,” Neurocomput, vol. 406, pp. 354–360, 2020.
  24. S. Kabir, L. Farrokhvar, and A. Dabouei, “A weakly supervised approach for thoracic diseases detection,” Expert Syst Appl, vol. 213, p. 118942, 2023.
  25. H. Chen, S. Miao, D. Xu, G. D. Hager, and A. P. Harrison, “Deep hiearchical multi-label classification applied to chest x-ray abnormality taxonomies,” Med Image Anal, vol. 66, p. 101811, 2020.
  26. H. H. Pham, T. T. Le, D. Q. Tran, D. T. Ngo, and H. Q. Nguyen, “Interpreting chest x-rays via cnns that exploit hierarchical disease dependencies and uncertainty labels,” Neurocomput, vol. 437, pp. 186–194, 2021.
  27. R. T. Mullapudi, W. R. Mark, N. Shazeer, and K. Fatahalian, “Hydranets: Specialized dynamic architectures for efficient inference,” in IEEE Comp Vis Pattern Recognit, 2018, pp. 8080–8089.
  28. B. Velasco, J. Cerquides, and J. L. Arcos, “Hydranet: A neural network for the estimation of multi-valued treatment effects,” in NeurIPS 2022 Workshop on Causality for Real-world Impact, 2018.
  29. A. Kumar, Y.-Y. Wang, K.-C. Liu, I.-C. Tsai, C.-C. Huang, and N. Hung, “Distinguishing normal and pulmonary edema chest x-ray using gabor filter and svm,” in IEEE ISBB 2014.   IEEE, 2014, pp. 1–4.
  30. F. H. O. Alfadhli, A. A. Mand, M. S. Sayeed, K. S. Sim, and M. Al-Shabi, “Classification of tuberculosis with surf spatial pyramid features,” in ICORAS.   IEEE, 2017, pp. 1–5.
  31. P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan et al., “Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning,” arXiv:1711.05225, 2017.
  32. J. Rubin, D. Sanghavi, C. Zhao, K. Lee, A. Qadir, and M. Xu-Wilson, “Large scale automated reading of frontal and lateral chest x-rays using dual convolutional neural networks,” arXiv:1804.07839, 2018.
  33. I. M. Baltruschat, H. Nickisch, M. Grass, T. Knopp, and A. Saalbach, “Comparison of deep learning approaches for multi-label chest x-ray classification,” Sci Rep, vol. 9, no. 1, pp. 1–10, 2019.
  34. L. Yao, E. Poblenz, D. Dagunts, B. Covington, D. Bernard, and K. Lyman, “Learning to diagnose from scratch by exploiting dependencies among labels,” arXiv:1710.10501, 2017.
  35. B. Chen, J. Li, X. Guo, and G. Lu, “Dualchexnet: dual asymmetric feature learning for thoracic disease classification in chest x-rays,” Biomed Sig Process Cont, vol. 53, p. 101554, 2019.
  36. M. Yang, H. Tanaka, and T. Ishida, “Performance improvement in multi-label thoracic abnormality classification of chest x-rays with noisy labels,” Int J Comput Assist Radiol Surg, vol. 18, no. 1, pp. 181–189, 2023.
  37. R. Hermoza, G. Maicas, J. C. Nascimento, and G. Carneiro, “Region proposals for saliency map refinement for weakly-supervised disease localisation and classification,” in Med Image Comput Comput Assist Inter.   Springer, 2020, pp. 539–549.
  38. B. Chen, Z. Zhang, Y. Li, G. Lu, and D. Zhang, “Multi-label chest x-ray image classification via semantic similarity graph embedding,” IEEE Trans Circuits Syst Video Technol, vol. 32, no. 4, pp. 2455–2468, 2022.
  39. B. Chen, J. Li, G. Lu, H. Yu, and D. Zhang, “Label co-occurrence learning with graph convolutional networks for multi-label chest x-ray image classification,” IEEE J Biomed Health Inf, vol. 24, no. 8, pp. 2292–2302, 2020.
  40. A. Güngör, S. U. Dar, c. Öztürk, Y. Korkmaz, G. Elmas, M. Özbey et al., “Adaptive diffusion priors for accelerated mri reconstruction,” arXiv:2207.05876, 2022.
  41. M. Özbey, O. Dalmaz, S. U. Dar, H. A. Bedel, c. Özturk, A. Güngör et al., “Unsupervised medical image translation with adversarial diffusion models,” arXiv:2207.08208, 2022.
  42. P. Chambon, C. Bluethgen, C. P. Langlotz, and A. Chaudhari, “Adapting pretrained vision-language foundational models to medical imaging domains,” arXiv:2210.04133, 2022.
  43. K. Packhäuser, L. Folle, F. Thamm, and A. Maier, “Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality classification systems,” arXiv:2211.01323, 2022.
  44. C. Yan, J. Yao, R. Li, Z. Xu, and J. Huang, “Weakly supervised deep learning for thoracic disease classification and localization on chest x-rays,” in ACM Int Conf Bioinf Comput Biol Health Inf, 2018, pp. 103–110.
  45. G. Zhao, S. Shao, and M. Yu, “Key techniques for classification of thorax diseases based on deep learning,” Int J Imaging Syst Tech, vol. 32, no. 6, pp. 2184–2197, 2022.
  46. B. Chen, Y. Lu, and G. Lu, “Multi-label chest x-ray image classification via label co-occurrence learning,” in PRCV.   Springer, 2019, pp. 682–693.
  47. A. I. Aviles-Rivero, N. Papadakis, R. Li, P. Sellars, Q. Fan, R. T. Tan et al., “Graphx^\\\backslash\small net-net-chest x-ray classification under extreme minimal supervision,” in Med Image Comput Comput Assist Inter.   Springer, 2019, pp. 504–512.
  48. B. Zhou, Y. Li, and J. Wang, “A weakly supervised adaptive densenet for classifying thoracic diseases and identifying abnormalities,” arXiv:1807.01257, 2018.
  49. Z. Ge, D. Mahapatra, S. Sedai, R. Garnavi, and R. Chakravorty, “Chest x-rays classification: A multi-label and fine-grained problem,” arXiv:1807.07247, 2018.
  50. R. López-González, J. Sánchez-García, B. Fos-Guarinos, F. García-Castro, Á. Alberich-Bayarri, E. Soria-Olivas et al., “Automated chest radiographs triage reading by a deep learning referee network,” medRxiv, pp. 2021–06, 2021.
  51. O. Dalmaz, M. Yurt, and T. Çukur, “Resvit: Residual vision transformers for multimodal medical image synthesis,” IEEE Trans Med Imaging, vol. 41, no. 10, pp. 2598–2614, 2022.
  52. X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” in IEEE Comp Vis Pattern Recognit, 2017, pp. 3462–3471.
  53. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in IEEE Int Conf Comput Vis, 2017, pp. 618–626.
  54. H. A. Bedel, I. Sivgin, O. Dalmaz, S. U. Dar, and T. Çukur, “BolT: Fused window transformers for fMRI time series analysis,” Med Image Anal, vol. 88, p. 102841, 2023.
  55. A. Majkowska, S. Mittal, D. F. Steiner, J. J. Reicher, S. M. McKinney, G. E. Duggan et al., “Chest radiograph interpretation with deep learning models: Assessment with radiologist-adjudicated reference standards and population-adjusted evaluation,” Radiology, vol. 294, no. 2, pp. 421–431, 2020.
  56. Ş. Öztürk and T. Çukur, “Deep clustering via center-oriented margin free-triplet loss for skin lesion detection in highly imbalanced datasets,” IEEE J Biomed Health Inf, vol. 26, no. 9, pp. 4679–4690, 2022.
  57. Ş. Öztürk, E. Çelik, and T. Çukur, “Content-based medical image retrieval with opponent class adaptive margin loss,” Inf Sci, p. 118938, 2023.
  58. J. Yang, C. Li, X. Dai, and J. Gao, “Focal modulation networks,” NeurIPS, vol. 35, pp. 4203–4217, 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Şaban Öztürk (12 papers)
  2. M. Yiğit Turalı (2 papers)
  3. Tolga Çukur (48 papers)
Citations (6)
View on Semantic Scholar