Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Edge-aware Feature Aggregation Network for Polyp Segmentation (2309.10523v1)

Published 19 Sep 2023 in cs.CV

Abstract: Precise polyp segmentation is vital for the early diagnosis and prevention of colorectal cancer (CRC) in clinical practice. However, due to scale variation and blurry polyp boundaries, it is still a challenging task to achieve satisfactory segmentation performance with different scales and shapes. In this study, we present a novel Edge-aware Feature Aggregation Network (EFA-Net) for polyp segmentation, which can fully make use of cross-level and multi-scale features to enhance the performance of polyp segmentation. Specifically, we first present an Edge-aware Guidance Module (EGM) to combine the low-level features with the high-level features to learn an edge-enhanced feature, which is incorporated into each decoder unit using a layer-by-layer strategy. Besides, a Scale-aware Convolution Module (SCM) is proposed to learn scale-aware features by using dilated convolutions with different ratios, in order to effectively deal with scale variation. Further, a Cross-level Fusion Module (CFM) is proposed to effectively integrate the cross-level features, which can exploit the local and global contextual information. Finally, the outputs of CFMs are adaptively weighted by using the learned edge-aware feature, which are then used to produce multiple side-out segmentation maps. Experimental results on five widely adopted colonoscopy datasets show that our EFA-Net outperforms state-of-the-art polyp segmentation methods in terms of generalization and effectiveness.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (78)
  1. J. Silva, A. Histace, O. Romain, X. Dray, and B. Granado, “Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer,” International Journal of Computer Assisted Radiology and Surgery, vol. 9, no. 2, pp. 283–293, 2014, DOI: 10.1007/s11548-013-0926-3.
  2. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Munich, Germany: Springer, 2015, pp. 234–241, DOI: 10.1007/978-3-319-24574-4_28.
  3. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” IEEE Transactions on Medical Imaging, vol. 39, no. 6, pp. 1856–1867, 2019, DOI: 10.1109/TMI.2019.2959609.
  4. D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, and H. D. Johansen, “Resunet++: An advanced architecture for medical image segmentation,” in IEEE International Symposium on Multimedia.   San Diego, CA, USA: IEEE, 2019, pp. 225–2255, DOI: 10.1109/ISM46123.2019.00049.
  5. K. Patel, A. M. Bur, and G. Wang, “Enhanced u-net: A feature enhancement network for polyp segmentation,” in Proc. IEEE Int. Robots and Vision, 2021, pp. 181–188.
  6. R. Zhang, G. Li, Z. Li, S. Cui, D. Qian, and Y. Yu, “Adaptive context selection for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Lima, Peru: Springer, 2020, pp. 253–262, DOI: 10.1007/978-3-030-59725-2_25.
  7. T.-C. Nguyen, T.-P. Nguyen, G.-H. Diep, A.-H. Tran-Dinh, T. V. Nguyen, and M.-T. Tran, “Ccbanet: Cascading context and balancing attention for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Strasbourg, France: Springer, 2021, pp. 633–643, DOI: 10.1007/978-3-030-87193-2_60.
  8. B. Murugesan, K. Sarveswaran, S. M. Shankaranarayana, K. Ram, J. Joseph, and M. Sivaprakasam, “Psi-Net: Shape and boundary aware joint multi-task deep network for medical image segmentation,” in Engineering in Medicine and Biology Society.   Germany, Germany: IEEE, 2019, pp. 7223–7226, DOI: 10.1109/EMBC.2019.8857339.
  9. Y. Fang, C. Chen, Y. Yuan, and K.-y. Tong, “Selective feature aggregation network with area-boundary constraints for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Shenzhen, China: Springer, 2019, pp. 302–310, DOI: 10.1007/978-3-030-32239-7_34.
  10. D.-P. Fan, G.-P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, “Pranet: Parallel reverse attention network for polyp segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Intervent., 2020, pp. 263–273.
  11. S. Hwang, J. Oh, W. Tavanapong, J. Wong, and P. C. De Groen, “Polyp detection in colonoscopy video using elliptical shape feature,” in IEEE International Conference on Image Processing, vol. 2.   IEEE, 2007, pp. II–465.
  12. J. Bernal, J. Sánchez, and F. Vilarino, “Towards automatic polyp detection with a polyp appearance model,” Pattern Recognition, vol. 45, no. 9, pp. 3166–3182, 2012, DOI: 10.1016/j.patcog.2012.03.002.
  13. Y. Wang, W. Tavanapong, J. Wong, J. Oh, and P. C. De Groen, “Part-based multiderivative edge cross-sectional profiles for polyp detection in colonoscopy,” IEEE Journal of Biomedical and Health Informatics, vol. 18, no. 4, pp. 1379–1389, 2013.
  14. A. V. Mamonov, I. N. Figueiredo, P. N. Figueiredo, and Y.-H. R. Tsai, “Automated polyp detection in colon capsule endoscopy,” IEEE Transactions on Medical Imaging, vol. 33, no. 7, pp. 1488–1502, 2014, DOI: 10.1109/TMI.2014.2314959.
  15. N. Tajbakhsh, S. R. Gurudu, and J. Liang, “Automated polyp detection in colonoscopy videos using shape and context information,” IEEE Transactions on Medical Imaging, vol. 35, no. 2, pp. 630–644, 2016, DOI: 10.1109/TMI.2015.2487997.
  16. Q. Li, G. Yang, Z. Chen, B. Huang, L. Chen, D. Xu, and OTHERS, “Colorectal polyp segmentation using a fully convolutional neural network,” in IEEE International Congress on Image and Signal Processing, Biomedical Engineering and Informatics.   IEEE, 2017, pp. 1–5.
  17. D. Jha, P. H. Smedsrud, M. A. Riegler, D. Johansen, T. De Lange, P. Halvorsen, and H. D. Johansen, “Resunet++: An advanced architecture for medical image segmentation,” in IEEE International Symposium on Multimedia, 2019, pp. 225–2255.
  18. Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: Redesigning skip connections to exploit multiscale features in image segmentation,” IEEE Transactions on Medical Imaging, vol. 39, no. 6, pp. 1856–1867, 2019.
  19. N. Ibtehaz and M. S. Rahman, “Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation,” Neural Networks, vol. 121, pp. 74–87, 2020.
  20. Z. Zhang, Q. Liu, and Y. Wang, “Road extraction by deep residual u-net,” IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 749–753, 2018.
  21. X. Li, H. Chen, X. Qi, Q. Dou, C.-W. Fu, and P.-A. Heng, “H-DenseUNet: hybrid densely connected unet for liver and tumor segmentation from ct volumes,” IEEE Transactions on Medical Imaging, vol. 37, no. 12, pp. 2663–2674, 2018.
  22. O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz et al., “Attention u-net: Learning where to look for the pancreas,” arXiv preprint arXiv:1804.03999, 2018.
  23. T. Zhou, Y. Zhou, K. He, C. Gong, J. Yang, H. Fu, and D. Shen, “Cross-level feature aggregation network for polyp segmentation,” Pattern Recognition, vol. 140, p. 109555, 2023.
  24. G. Yue, W. Han, B. Jiang, T. Zhou, R. Cong, and T. Wang, “Boundary constraint network with cross layer feature integration for polyp segmentation,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 8, pp. 4090–4099, 2022.
  25. F. Liu, Z. Hua, J. Li, and L. Fan, “Dbmf: Dual branch multiscale feature fusion network for polyp segmentation,” Computers in Biology and Medicine, vol. 151, p. 106304, 2022.
  26. Y. Su, J. Cheng, C. Zhong, C. Jiang, J. Ye, and J. He, “Accurate polyp segmentation through enhancing feature fusion and boosting boundary performance,” Neurocomputing, vol. 545, p. 126233, 2023.
  27. P. Song, J. Li, and H. Fan, “Attention based multi-scale parallel network for polyp segmentation,” Computers in Biology and Medicine, vol. 146, p. 105476, 2022.
  28. N. K. Tomar, D. Jha, U. Bagci, and S. Ali, “Tganet: text-guided attention for improved polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Springer, 2022, pp. 151–160.
  29. J. Wei, Y. Hu, R. Zhang, Z. Li, S. K. Zhou, and S. Cui, “Shallow attention network for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Strasbourg, France: Springer, 2021, pp. 699–708, DOI: 10.1007/978-3-030-87193-2_66.
  30. H. Wu, Z. Zhao, J. Zhong, W. Wang, Z. Wen, and J. Qin, “Polypseg+: A lightweight context-aware network for real-time polyp segmentation,” IEEE Transactions on Cybernetics, 2022.
  31. Z. Yin, K. Liang, Z. Ma, and J. Guo, “Duplex contextual relation network for polyp segmentation,” in Proc. Int. Symp. Biomed. Imaging, 2022, pp. 1–5.
  32. Y. Lin, J. Wu, G. Xiao, J. Guo, G. Chen, and J. Ma, “Bsca-net: Bit slicing context attention network for polyp segmentation,” Pattern Recognition, vol. 132, p. 108917, 2022.
  33. H. Ding, X. Jiang, A. Q. Liu, N. M. Thalmann, and G. Wang, “Boundary-aware feature propagation for scene segmentation,” in Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 6819–6829.
  34. X. Qin, Z. Zhang, C. Huang, C. Gao, M. Dehghan, and M. Jagersand, “Basnet: Boundary-aware salient object detection,” in Conference on Computer Vision and Pattern Recognition, 2019, pp. 7479–7489.
  35. K. Liu, Y. Zhao, Q. Nie, Z. Gao, and B. M. Chen, “Weakly supervised 3d scene segmentation with region-level boundary awareness and instance discrimination,” in European Conference on Computer V ision.   Springer, 2022, pp. 37–55.
  36. T. Zhou, Y. Zhou, C. Gong, J. Yang, and Y. Zhang, “Feature aggregation and propagation network for camouflaged object detection,” IEEE Trans. Image Process., vol. 31, pp. 7036–7047, 2022.
  37. F. Li, X. Du, L. Zhang, and A. Liu, “Image feature fusion method based on edge detection,” Information Technology and Control, vol. 52, no. 1, pp. 5–24, 2023.
  38. M. Nawaz, T. Nazir, M. Masood, F. Ali, M. A. Khan, U. Tariq, N. Sahar, and R. Damaševičius, “Melanoma segmentation: A framework of improved densenet77 and unet convolutional neural network,” International Journal of Imaging Systems and Technology, vol. 32, no. 6, pp. 2137–2153, 2022.
  39. R. Maskeliunas, R. Damasevicius, D. Vitkute-Adzgauskiene, and S. Misra, “Pareto optimized large mask approach for efficient and background humanoid shape removal,” IEEE access, 2023.
  40. H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P.-A. Heng, “Dcan: Deep contour-aware networks for object instance segmentation from histology images,” Medical Image Analysis, vol. 36, pp. 135–146, 2017.
  41. Z. Zhang, H. Fu, H. Dai, J. Shen, Y. Pang, and L. Shao, “Et-net: A generic edge-attention guidance network for medical image segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Springer, 2019, pp. 442–450.
  42. D.-P. Fan, G.-P. Ji, T. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, “Pranet: Parallel reverse attention network for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Lima, Peru: Springer, 2020, pp. 263–273, DOI: https://doi.org/10.1007/978-3-030-59725-2_26.
  43. L. K. Ramasamy, S. G. Padinjappurathu, S. Kadry, and R. Damaševičius, “Detection of diabetic retinopathy using a fusion of textural and ridgelet features of retinal images and sequential minimal optimization classifier,” PeerJ computer science, vol. 7, p. e456, 2021.
  44. R. Wang, S. Chen, C. Ji, J. Fan, and Y. Li, “Boundary-aware context neural network for medical image segmentation,” Medical Image Analysis, vol. 78, p. 102395, 2022, DOI: 10.1016/j.media.2022.102395.
  45. K. Wang, X. Zhang, X. Zhang, Y. Lu, S. Huang, and D. Yang, “EANet: Iterative edge attention network for medical image segmentation,” Pattern Recognition, vol. 127, p. 108636, 2022.
  46. Y. Kim, B.-N. Kang, and D. Kim, “San: Learning relationship between convolutional features for multi-scale object detection,” in Proceedings of the European Conference on Computer Vision, 2018, pp. 316–331.
  47. J. Cao, Y. Pang, S. Zhao, and X. Li, “High-level semantic networks for multi-scale object detection,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 10, pp. 3372–3386, 2019.
  48. T. Zhou, D.-P. Fan, G. Chen, Y. Zhou, and H. Fu, “Specificity-preserving rgb-d saliency detection,” Computational Visual Media, vol. 9, pp. 297–317, 2023.
  49. J. Li, F. Fang, K. Mei, and G. Zhang, “Multi-scale residual network for image super-resolution,” in Proceedings of the European Conference on Computer Vision, 2018, pp. 517–532.
  50. J. Li, F. Fang, J. Li, K. Mei, and G. Zhang, “MDCN: Multi-scale dense cross network for image super-resolution,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 7, pp. 2547–2561, 2020.
  51. J. He, Z. Deng, and Y. Qiao, “Dynamic multi-scale filters for semantic segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 3562–3572.
  52. J. Gu, H. Kwon, D. Wang, W. Ye, M. Li, Y.-H. Chen, L. Lai, V. Chandra, and D. Z. Pan, “Multi-scale high-resolution vision transformer for semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 12 094–12 103.
  53. H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2017, pp. 2881–2890.
  54. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, 2017.
  55. H. Yang, W. Huang, K. Qi, C. Li, X. Liu, M. Wang, H. Zheng, and S. Wang, “CLCI-Net: Cross-level fusion and context inference networks for lesion segmentation of chronic stroke,” in Medical Image Computing and Computer Assisted Intervention.   Springer, 2019, pp. 266–274.
  56. X. Zhao, L. Zhang, and H. Lu, “Automatic polyp segmentation via multi-scale subtraction network,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Strasbourg, France: Springer, 2021, pp. 120–130, DOI: 10.1007/978-3-030-87193-2_12.
  57. A. Srivastava, D. Jha, S. Chanda, U. Pal, H. D. Johansen et al., “Msrf-net: A multi-scale residual fusion network for biomedical image segmentation,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 5, pp. 2252–2263, 2021.
  58. H. Yang, T. Zhou, Y. Zhou, Y. Zhang, and H. Fu, “Flexible fusion network for multi-modal brain tumor segmentation,” IEEE Journal of Biomedical and Health Informatics, 2023.
  59. S.-H. Gao, M.-M. Cheng, K. Zhao, X.-Y. Zhang, M.-H. Yang, and P. Torr, “Res2net: A new multi-scale backbone architecture,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 2, pp. 652–662, 2019, DOI: 10.1109/TPAMI.2019.2938758.
  60. D.-P. Fan, T. Zhou, G.-P. Ji, Y. Zhou, G. Chen, H. Fu, J. Shen, and L. Shao, “Inf-net: Automatic covid-19 lung infection segmentation from ct images,” IEEE Transactions on Medical Imaging, vol. 39, no. 8, pp. 2626–2637, 2020, DOI: 10.1109/TMI.2020.2996645.
  61. Y. Sun, S. Wang, C. Chen, and T.-Z. Xiang, “Boundary-guided camouflaged object detection,” in International Joint Conference on Artificial Intelligence, 2022, pp. 1335–1341.
  62. J.-X. Zhao, J.-J. Liu, D.-P. Fan, Y. Cao, J. Yang, and M.-M. Cheng, “EGNet: Edge guidance network for salient object detection,” in International Conference on Computer Vision, 2019, pp. 8779–8788.
  63. Y. Dai, F. Gieseke, S. Oehmcke, Y. Wu, and K. Barnard, “Attentional feature fusion,” in IEEE WACV, 2021, pp. 3560–3569.
  64. J. Wei, S. Wang, and Q. Huang, “F3Net: Fusion, Feedback and Focus for Salient Object Detection,” in AAAI Conference on Artificial Intelligence, 2020, pp. 12 321–12 328.
  65. J. Bernal, F. J. Sánchez, G. Fernández-Esparrach, D. Gil, C. Rodríguez, and F. Vilariño, “Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians,” Computerized Medical Imaging and Graphics, vol. 43, pp. 99–111, 2015, DOI: 10.1016/j.compmedimag.2015.02.007.
  66. D. Jha, P. H. Smedsrud, M. A. Riegler, P. Halvorsen, T. de Lange, D. Johansen, and H. D. Johansen, “Kvasir-seg: A segmented polyp dataset,” in International Conference on Multimedia Modeling.   Daejeon, Korea: Springer, 2020, pp. 451–462, DOI: 10.1007/978-3-030-37734-2_37.
  67. C.-H. Huang, H.-Y. Wu, and Y.-L. Lin, “Hardnet-mseg: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps,” arXiv preprint arXiv:2101.07172, 2021.
  68. R. Zhang, P. Lai, X. Wan, D.-J. Fan, F. Gao, X.-J. Wu, and G. Li, “Lesion-aware dynamic kernel for polyp segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention.   Springer, 2022, pp. 99–109.
  69. D. Vázquez, J. Bernal, F. J. Sánchez, G. Fernández-Esparrach, A. M. López, A. Romero, M. Drozdzal, and A. Courville, “A benchmark for endoluminal scene segmentation of colonoscopy images,” Journal of Healthcare Engineering, vol. 2017, p. 4037190, 2017, DOI: 10.1155/2017/4037190.
  70. C. Yang, X. Guo, M. Zhu, B. Ibragimov, and Y. Yuan, “Mutual-prototype adaptation for cross-domain polyp segmentation,” IEEE J. Biomed. Health Informat., vol. 25, no. 10, pp. 3886–3897, 2021.
  71. T. Zhou, D.-P. Fan, M.-M. Cheng, J. Shen, and L. Shao, “RGB-D salient object detection: A survey,” Computational Visual Media, vol. 7, no. 1, pp. 37–69, 2021.
  72. D.-P. Fan, G.-P. Ji, G. Sun, M.-M. Cheng, J. Shen, and L. Shao, “Camouflaged object detection,” in Conference on Computer Vision and Pattern Recognition.   Seattle, WA, USA: IEEE, 2020, pp. 2777–2787, DOI: 10.1109/CVPR42600.2020.00285.
  73. D.-P. Fan, M.-M. Cheng, Y. Liu, T. Li, and A. Borji, “Structure-measure: A new way to evaluate foreground maps,” in International Conference on Computer Vision.   Venice, Italy: IEEE, 2017, pp. 4548–4557, DOI: 10.1109/ICCV.2017.487.
  74. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in Conference on Computer Vision and Pattern Recognition.   Miami, FL, USA: IEEE, 2009, pp. 1597–1604, DOI: 10.1109/CVPR.2009.5206596.
  75. D.-P. Fan, C. Gong, Y. Cao, B. Ren, M.-M. Cheng, and A. Borji, “Enhanced-alignment measure for binary foreground map evaluation,” in International Joint Conference on Artificial Intelligence.   Stockholm, Sweden: IJCAI, 2018, pp. 698–704, DOI: 10.24963/ijcai.2018/97.
  76. Y. Li, Q. Hou, Z. Zheng, M.-M. Cheng, J. Yang, and X. Li, “Large selective kernel network for remote sensing object detection,” arXiv preprint arXiv:2303.09030, 2023.
  77. P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz, “Importance estimation for neural network pruning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 11 264–11 272.
  78. J. Gou, B. Yu, S. J. Maybank, and D. Tao, “Knowledge distillation: A survey,” International Journal of Computer Vision, vol. 129, pp. 1789–1819, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Tao Zhou (398 papers)
  2. Yizhe Zhang (127 papers)
  3. Geng Chen (115 papers)
  4. Yi Zhou (438 papers)
  5. Ye Wu (39 papers)
  6. Deng-Ping Fan (88 papers)
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now