A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition (2309.13578v1)
Abstract: Panoptic segmentation in agriculture is an advanced computer vision technique that provides a comprehensive understanding of field composition. It facilitates various tasks such as crop and weed segmentation, plant panoptic segmentation, and leaf instance segmentation, all aimed at addressing challenges in agriculture. Exploring the application of panoptic segmentation in agriculture, the 8th Workshop on Computer Vision in Plant Phenotyping and Agriculture (CVPPA) hosted the challenge of hierarchical panoptic segmentation of crops and weeds using the PhenoBench dataset. To tackle the tasks presented in this competition, we propose an approach that combines the effectiveness of the Segment AnyThing Model (SAM) for instance segmentation with prompt input from object detection models. Specifically, we integrated two notable approaches in object detection, namely DINO and YOLO-v8. Our best-performing model achieved a PQ+ score of 81.33 based on the evaluation metrics of the competition.
- Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12475–12485, 2020.
- Masked-attention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1290–1299, 2022.
- Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
- Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
- Yolo by ultralytics. URL: https://github. com/ultralytics/ultralytics, 2023.
- Segment anything in high quality. arXiv preprint arXiv:2306.01567, 2023.
- Segment anything. arXiv preprint arXiv:2304.02643, 2023.
- detrex: Benchmarking detection transformers, 2023.
- Hierarchical approach for joint semantic, plant instance, and leaf instance segmentation in the agricultural domain. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9601–9607. IEEE, 2023.
- When sam meets sonar images. arXiv preprint arXiv:2306.14109, 2023.
- Phenobench – a large dataset and benchmarks for semantic image interpretation in the agricultural domain, 2023.
- Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.