Papers
Topics
Authors
Recent
2000 character limit reached

Polyper: Boundary Sensitive Polyp Segmentation (2312.08735v1)

Published 14 Dec 2023 in cs.CV

Abstract: We present a new boundary sensitive framework for polyp segmentation, called Polyper. Our method is motivated by a clinical approach that seasoned medical practitioners often leverage the inherent features of interior polyp regions to tackle blurred boundaries.Inspired by this, we propose explicitly leveraging polyp regions to bolster the model's boundary discrimination capability while minimizing computation. Our approach first extracts boundary and polyp regions from the initial segmentation map through morphological operators. Then, we design the boundary sensitive attention that concentrates on augmenting the features near the boundary regions using the interior polyp regions's characteristics to generate good segmentation results. Our proposed method can be seamlessly integrated with classical encoder networks, like ResNet-50, MiT-B1, and Swin Transformer. To evaluate the effectiveness of Polyper, we conduct experiments on five publicly available challenging datasets, and receive state-of-the-art performance on all of them. Code is available at https://github.com/haoshao-nku/medical_seg.git.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint arXiv:1802.06955.
  2. Impact of artificial intelligence on colorectal polyp detection. Best Practice & Research Clinical Gastroenterology, 52: 101713.
  3. SegT: A Novel Separated Edge-guidance Transformer Network for Polyp Segmentation. arXiv preprint arXiv:2306.10773.
  4. Reverse attention for salient object detection. In Proceedings of the European conference on computer vision (ECCV), 234–250.
  5. Contributors, M. 2020. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.
  6. Rates of incomplete resection of 1-to 20-mm colorectal polyps: a systematic review and meta-analysis. Gastroenterology, 159(3): 904–914.
  7. Polyp-pvt: Polyp segmentation with pyramid vision transformers. arXiv preprint arXiv:2108.06932.
  8. Pranet: Parallel reverse attention network for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23, 263–273. Springer.
  9. RFENet: Towards Reciprocal Feature Evolution for Glass Segmentation. arXiv preprint arXiv:2307.06099.
  10. Selective feature aggregation network with area-boundary constraints for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22, 302–310. Springer.
  11. Is attention better than matrix decomposition? arXiv preprint arXiv:2109.04553.
  12. Digital image processing. Addison-Wesley Longman Publishing Co., Inc.
  13. Enhanced boundary learning for glass-like object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 15859–15868.
  14. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  15. Hardnet-mseg: A simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps. arXiv preprint arXiv:2101.07172.
  16. RTNet: relation transformer network for diabetic retinopathy multi-lesion segmentation. IEEE Transactions on Medical Imaging, 41(6): 1596–1607.
  17. AttResDU-Net: Medical Image Segmentation Using Attention-based Residual Double U-Net. arXiv preprint arXiv:2306.14255.
  18. Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In ICLR.
  19. Medical image segmentation using squeeze-and-expansion transformers. arXiv preprint arXiv:2105.09511.
  20. Ds-transunet: Dual swin transformer u-net for medical image segmentation. IEEE Transactions on Instrumentation and Measurement, 71: 1–15.
  21. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2117–2125.
  22. FTMF-Net: A Fourier Transform-Multiscale Feature Fusion Network For Segmentation Of Small Polyp Objects. IEEE Transactions on Instrumentation and Measurement.
  23. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012–10022.
  24. CaraNet: context axial reverse attention network for segmentation of small medical objects. In Medical Imaging 2022: Image Processing, volume 12032, 81–92. SPIE.
  25. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.
  26. SwinE-Net: hybrid deep learning approach to novel polyp segmentation using convolutional neural network and Swin Transformer. Journal of Computational Design and Engineering, 9(2): 616–632.
  27. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32.
  28. Growth rates and histopathological outcomes of small (6–9 mm) colorectal polyps based on CT colonography surveillance and endoscopic removal. Gut.
  29. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 234–241. Springer.
  30. Edge-Aware Mirror Network for Camouflaged Object Detection. arXiv preprint arXiv:2307.03932.
  31. ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentation. arXiv e-prints, arXiv–2205.
  32. Kiu-net: Towards accurate segmentation of biomedical images using over-complete representations. In International conference on Medical Image Computing and Computer-Assisted Intervention, 363–373. Springer.
  33. Attention is all you need. Advances in neural information processing systems, 30.
  34. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7794–7803.
  35. SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation. arXiv preprint arXiv:2307.12591.
  36. Shallow attention network for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, 699–708. Springer.
  37. Precise yet efficient semantic calibration and refinement in convnets for real-time polyp segmentation from colonoscopy videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 2916–2924.
  38. Weighted res-unet for high-quality retina vessel segmentation. In International Conference on Information Technology in Medicine and Education (ITME), 327–331. IEEE.
  39. Segmenting transparent objects in the wild. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16, 696–711. Springer.
  40. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems, 34: 12077–12090.
  41. Camoformer: Masked separable attention for camouflaged object detection. arXiv preprint arXiv:2212.06570.
  42. 3D Medical Image Segmentation based on multi-scale MPU-Net. arXiv preprint arXiv:2307.05799.
  43. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5728–5739.
  44. Adaptive context selection for polyp segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part VI 23, 253–262. Springer.
  45. Transfuse: Fusing transformers and cnns for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24, 14–24. Springer.
  46. Unet++: A nested u-net architecture for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 3–11. Springer.
  47. The optimal connection model for blood vessels segmentation and the MEA-Net. arXiv preprint arXiv:2306.01808.
Citations (8)
View on Semantic Scholar

Summary

  • The paper introduces a two-stage decoder that first extracts potential boundaries and then refines them using a boundary-sensitive attention module.
  • It leverages morphological operators and clinical strategies to improve the identification of blurred polyp boundaries.
  • Experimental results demonstrate superior performance with enhanced Dice and IoU metrics, especially for small, early-stage polyps.

Introduction

Colon polyps are an important indicator for the risk of colon cancer, but diagnosing them during colonoscopy can be challenging due to their varied appearances and often indistinct boundaries. Identifying polyp boundaries precisely is critical for accurate diagnostics and subsequent treatment. In medical practice, distinguishing blurred boundaries is one of the significant obstacles when it comes to early-stage polyps that lack clear contrast. Existing methods have addressed polyp segmentation, but most face difficulties particularly with small or early-stage polyps.

Method Overview

In response to these challenges, a new framework called "Polyper" has been developed which introduces a boundary-sensitive polyp segmentation approach. This method is influenced by clinical strategies used by experienced practitioners who leverage features from within the polyps to identify their boundaries. Polyper applies an innovative two-stage decoder comprised of a potential boundary extraction stage followed by a boundary sensitive refinement stage.

In the initial stage, an interior segmentation map is generated from which potential boundary and non-boundary regions of polyps are extracted using morphological operators. Subsequently, the boundary-sensitive attention module refines the boundary regions using the characteristics of the non-boundary regions.

Experimental Evaluation

The performance of Polyper was rigorously evaluated across five challenging, publicly available datasets. The outcomes of these experiments confirmed that Polyper achieves state-of-the-art results on all five datasets, offering superior performance in terms of both mean Dice coefficient and mean Intersection over Union metrics. Importantly, Polyper demonstrated strong performance even with smaller polyps that are typically challenging to segment accurately.

Contributions and Further Work

The key contributions of this approach include the boundary sensitive attention module which effectively models relationships within the polyp regions to improve boundary delineation. Additionally, the novel two-stage decoder aids in correctly identifying polyp boundaries, overcoming the issue of boundary blurring commonly encountered during endoscopic procedures.

Despite its strengths, Polyper does have limitations. It assumes polyp localization is accurate and may not handle false positives or negatives effectively. Also, the current strategy uses a fixed width to define boundary regions, which might not sufficiently account for variation in polyp characteristics. Future work may center on developing methods for adaptive boundary width determination.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown

Whiteboard

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up