Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Scalable Semantic 3D Mapping of Coral Reefs with Deep Learning (2309.12804v1)

Published 22 Sep 2023 in cs.CV

Abstract: Coral reefs are among the most diverse ecosystems on our planet, and are depended on by hundreds of millions of people. Unfortunately, most coral reefs are existentially threatened by global climate change and local anthropogenic pressures. To better understand the dynamics underlying deterioration of reefs, monitoring at high spatial and temporal resolution is key. However, conventional monitoring methods for quantifying coral cover and species abundance are limited in scale due to the extensive manual labor required. Although computer vision tools have been employed to aid in this process, in particular SfM photogrammetry for 3D mapping and deep neural networks for image segmentation, analysis of the data products creates a bottleneck, effectively limiting their scalability. This paper presents a new paradigm for mapping underwater environments from ego-motion video, unifying 3D mapping systems that use machine learning to adapt to challenging conditions under water, combined with a modern approach for semantic segmentation of images. The method is exemplified on coral reefs in the northern Gulf of Aqaba, Red Sea, demonstrating high-precision 3D semantic mapping at unprecedented scale with significantly reduced required labor costs: a 100 m video transect acquired within 5 minutes of diving with a cheap consumer-grade camera can be fully automatically analyzed within 5 minutes. Our approach significantly scales up coral reef monitoring by taking a leap towards fully automatic analysis of video transects. The method democratizes coral reef transects by reducing the labor, equipment, logistics, and computing cost. This can help to inform conservation policies more efficiently. The underlying computational method of learning-based Structure-from-Motion has broad implications for fast low-cost mapping of underwater environments other than coral reefs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. AgiSoft. Metashape professional (version 1.8.3). http://www.agisoft.com/downloads/installer/, 2022.
  2. Coralseg: Learning coral segmentation from sparse annotations. Journal of Field Robotics, 36(8):1456–1477, 2019.
  3. High-resolution reef bathymetry and coral habitat complexity from airborne imaging spectroscopy. Remote Sensing, 12(2):310, 2020.
  4. Automated annotation of coral reef survey images. In 2012 IEEE conference on computer vision and pattern recognition, pages 1170–1177. IEEE, 2012.
  5. Towards automated annotation of benthic survey images: Variability of human experts and operational modes of automation. PloS one, 10(7):e0130312, 2015.
  6. Risk-sensitive planning for conserving coral reefs under rapid climate change. Conservation Letters, 11(6):e12587, 2018.
  7. Unsupervised scale-consistent depth learning from video. International Journal of Computer Vision, 129(9):2548–2564, 2021.
  8. Reefscape genomics: leveraging advances in 3d imaging to assess fine-scale patterns of genomic variation on coral reefs. Frontiers in Marine Science, page 875, 2021.
  9. Integrating structure-from-motion photogrammetry with geospatial software as a novel technique for quantifying 3d ecological characteristics of coral reefs. PeerJ, 3:e1077, 2015.
  10. JH Carleton and TJ Done. Quantitative video sampling of coral reef benthos: large-scale application. Coral Reefs, 14:35–46, 1995.
  11. A new deep learning engine for coralnet. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3693–3702, 2021a.
  12. Overlapnet: Loop closing for lidar-based slam. arXiv preprint arXiv:2105.11344, 2021b.
  13. A diver-operated hyperspectral imaging and topographic surveying system for automated mapping of benthic habitats. Scientific reports, 7(1):7122, 2017.
  14. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 303–312, 1996.
  15. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  16. Future loss of local-scale thermal refugia in coral reef ecosystems. Plos Climate, 1(2):e0000004, 2022.
  17. A coral reef refuge in the red sea. Global change biology, 19(12):3640–3647, 2013.
  18. Species richness on coral reefs and the pursuit of convergent global estimates. Current Biology, 25(4):500–505, 2015.
  19. Mapping submerged habitats and mangroves of lampi island marine national park (myanmar) from in situ and satellite observations. Remote Sensing, 8(1):2, 2015.
  20. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  21. Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks. PloS one, 15(3):e0230671, 2020.
  22. Global warming and recurrent mass bleaching of corals. Nature, 543(7645):373–377, 2017.
  23. El21-estonia: Report of the ms estonia shipwreck site survey with rv electra, 2021.
  24. Aide: Accelerating image-based ecological surveys with interactive machine learning. Methods in Ecology and Evolution, 11(12):1716–1727, 2020.
  25. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
  26. Nancy Knowlton and Jeremy B C Jackson. Shifting baselines, local impacts, and global change on coral reefs. PLoS biology, 6(2):e54, 2008.
  27. Common reef-building coral in the northern red sea resistant to elevated temperature and acidification. Royal Society open science, 4(5):170038, 2017.
  28. Measuring coral reef terrain roughness using ‘structure-from-motion’close-range photogrammetry. Geomorphology, 242:21–28, 2015.
  29. Deconstructing self-supervised monocular reconstruction: The design decisions that matter. arXiv preprint arXiv:2208.01489, 2022.
  30. Global warming of 1.5 c. An IPCC Special Report on the impacts of global warming of, 1(5), 2018.
  31. Climate change 2021: the physical science basis. Contribution of working group I to the sixth assessment report of the intergovernmental panel on climate change, page 2, 2021.
  32. NOAA. Fact sheet: Coral reefs. https://www.coast.noaa.gov/states/fast-facts/coral-reefs.html, 2022. Accessed: 2022-09-09.
  33. Coral reef monitoring, reef assessment technologies, and ecosystem-based management. Frontiers in Marine Science, 6:580, 2019.
  34. Thermal refugia against coral bleaching throughout the northern red sea. Global change biology, 24(2):e474–e484, 2018.
  35. Status of coral reef monitoring: An assessment of methods and data at the national level. International Coral Reef Initiative, 2021.
  36. Gopros™ as an underwater photogrammetry tool for citizen science. PeerJ, 4:e1960, 2016.
  37. How reliable is structure from motion (sfm) over time and between observers? a case study using coral reef bommies. Remote Sensing, 9(7):740, 2017.
  38. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.
  39. Fast and pervasive transcriptomic resilience and acclimation of extremely heat-tolerant coral holobionts from the northern red sea. Proceedings of the National Academy of Sciences, 118(19):e2023298118, 2021.
  40. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  41. Digitizing the coral reef: Machine learning of underwater spectral images enables dense taxonomic mapping of benthic habitats. Methods in Ecology and Evolution, 14(2):596–613, 2023.
  42. Status of coral reefs of the world: 2020. Global Coral Reef Monitoring Network, 2021.
  43. End of the chain? rugosity and fine-scale bathymetry from existing underwater digital imagery using structure-from-motion (sfm) technology. Coral Reefs, 35(3):889–894, 2016.
  44. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on computer vision, pages 402–419. Springer, 2020.
  45. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in Neural Information Processing Systems, 34:16558–16569, 2021.
  46. Which method for which purpose? a comparison of line intercept transect and underwater photogrammetry methods for coral reef surveys. Frontiers in Marine Science, 8:636902, 2021.
  47. Contrasting heat stress response patterns of coral holobionts across the red sea suggest distinct mechanisms of thermal tolerance. Molecular ecology, 30(18):4466–4480, 2021.
  48. Leveraging automated image analysis tools to transform our capacity to assess status and trends of coral reefs. Frontiers in Marine Science, page 222, 2019.
  49. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500, 2017.
  50. Repeatable semantic reef-mapping through photogrammetry and label-augmentation. Remote Sensing, 13(4):659, 2021.
  51. Unsupervised learning of depth and ego-motion from video. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1851–1858, 2017.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now