TreeLearn: A Comprehensive Deep Learning Method for Segmenting Individual Trees from Ground-Based LiDAR Forest Point Clouds (2309.08471v2)
Abstract: Laser-scanned point clouds of forests make it possible to extract valuable information for forest management. To consider single trees, a forest point cloud needs to be segmented into individual tree point clouds. Existing segmentation methods are usually based on hand-crafted algorithms, such as identifying trunks and growing trees from them, and face difficulties in dense forests with overlapping tree crowns. In this study, we propose TreeLearn, a deep learning-based approach for tree instance segmentation of forest point clouds. Unlike previous methods, TreeLearn is trained on already segmented point clouds in a data-driven manner, making it less reliant on predefined features and algorithms. Furthermore, TreeLearn is implemented as a fully automatic pipeline and does not rely on extensive hyperparameter tuning, which makes it easy to use. Additionally, we introduce a new manually segmented benchmark forest dataset containing 156 full trees, and 79 partial trees, that have been cleanly segmented by hand. The data is generated by mobile laser scanning and contributes to create a larger and more diverse data basis for model development and fine-grained instance segmentation evaluation. We trained TreeLearn on forest point clouds of 6665 trees, labeled using the Lidar360 software. An evaluation on the benchmark dataset shows that TreeLearn performs equally well or better than the algorithm used to generate its training data. Furthermore, the method's performance can be vastly improved by fine-tuning on the cleanly labeled benchmark dataset. The TreeLearn code is available from https://github.com/ecker-lab/TreeLearn. The data as well as trained models can be found at https://doi.org/10.25625/VPMPID.
- “The use of a mobile laser scanning system for mapping large forest plots” In IEEE Geosci. Remote Sens. Lett. 11.9 IEEE, 2014, pp. 1504–1508 DOI: https://doi.org/10.1109/lgrs.2013.2297418
- Mathias Disney “Terrestrial LiDAR: a three-dimensional revolution in how we look at trees” In New Phytologist 222.4 Wiley Online Library, 2019, pp. 1736–1741 DOI: https://doi.org/10.1111/nph.15517
- “Terrestrial laser scanning in forest ecology: Expanding the horizon” In Remote Sens. Environ. 251 Elsevier, 2020, pp. 112102 DOI: https://doi.org/10.1016/j.rse.2020.112102
- “Tree species classification using structural features derived from terrestrial laser scanning” In ISPRS J. Photogramm. Remote Sens. 168 Elsevier, 2020, pp. 170–181 DOI: https://doi.org/10.1016/j.isprsjprs.2020.08.009
- “See the forest and the trees: Effective machine and deep learning algorithms for wood filtering and tree species classification from terrestrial laser scanning” In ISPRS J. Photogramm. Remote Sens. 168 Elsevier, 2020, pp. 1–16 DOI: https://doi.org/10.1016/j.isprsjprs.2020.08.001
- “Predicting tree species from 3D laser scanning point clouds using deep learning” In Frontiers Plant Sci. 12 Frontiers Media SA, 2021, pp. 635440 DOI: https://doi.org/10.3389/fpls.2021.635440
- “LiDAR applications to estimate forest biomass at individual tree scale: Opportunities, challenges and future perspectives” In Forests 12.5 MDPI, 2021, pp. 550 DOI: https://doi.org/10.3390/f12050550
- “Non-destructive estimation of individual tree biomass: Allometric models, terrestrial and UAV laser scanning” In Remote Sens. Environ. 280 Elsevier, 2022, pp. 113180 DOI: https://doi.org/10.1016/j.rse.2022.113180
- “A novel algorithm of individual tree crowns segmentation considering three-dimensional canopy attributes using UAV oblique photos” In Int. J. Appl. Earth Obs. Geoinformation 112 Elsevier, 2022, pp. 102893 DOI: https://doi.org/10.1016/j.jag.2022.102893
- “Benchmarking airborne laser scanning tree segmentation algorithms in broadleaf forests shows high accuracy only for canopy trees” In Int. J. Appl. Earth Obs. Geoinformation 123 Elsevier, 2023, pp. 103490 DOI: https://doi.org/10.1016/j.jag.2023.103490
- “3D Forest: An application for descriptions of three-dimensional forest structures using terrestrial LiDAR” In PloS one 12.5 Public Library of Science San Francisco, CA USA, 2017, pp. e0176871 DOI: https://doi.org/10.1371/journal.pone.0176871
- “Massive-scale tree modelling from TLS data” In ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci 2.3, 2015, pp. 189 DOI: https://doi.org/10.5194/isprsannals-ii-3-w4-189-2015
- Andrew Burt, Mathias Disney and Kim Calders “Extracting individual trees from lidar point clouds using treeseg” In Methods Ecol. Evol. 10.3 Wiley Online Library, 2019, pp. 438–445 DOI: https://doi.org/10.1111/2041-210x.13121
- “Segmenting individual tree from TLS point clouds using improved DBSCAN” In Forests 13.4 MDPI, 2022, pp. 566 DOI: https://doi.org/10.3390/f13040566
- “Segmentation of individual trees from TLS and MLS data” In IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10.2 IEEE, 2016, pp. 774–787 DOI: https://doi.org/10.1109/jstars.2016.2565519
- Johannes Heinzel and Markus O Huber “Constrained spectral clustering of individual trees in dense forest using terrestrial laser scanning data” In Remote Sensing 10.7 Multidisciplinary Digital Publishing Institute, 2018, pp. 1056 DOI: https://doi.org/10.3390/rs10071056
- “3D Graph-Based Individual-Tree Isolation (Treeiso) from Terrestrial Laser Scanning Point Clouds” In Remote Sensing 14.23 MDPI, 2022, pp. 6116 DOI: https://doi.org/10.3390/rs14236116
- “Individual tree extraction from terrestrial laser scanning data via graph pathing” In Forest Ecosystems 8 Springer, 2021, pp. 1–11 DOI: https://doi.org/10.1186/s40663-021-00340-w
- “Point-cloud segmentation of individual trees in complex natural forest scenes based on a trunk-growth method” In J. For. Res. 32.6 Springer, 2021, pp. 2403–2414 DOI: https://doi.org/10.1007/s11676-021-01303-1
- “Segmenting tree crowns from terrestrial and mobile LiDAR data by exploring ecological theories” In ISPRS J. Photogramm. Remote Sens. 110 Elsevier, 2015, pp. 66–76 DOI: https://doi.org/10.1016/j.isprsjprs.2015.10.007
- Di Wang “Unsupervised semantic and instance segmentation of forest point clouds” In ISPRS J. Photogramm. Remote Sens. 165 Elsevier, 2020, pp. 86–97 DOI: https://doi.org/10.1016/j.isprsjprs.2020.04.020
- “Topology-based individual tree segmentation for automated processing of terrestrial laser scanning point clouds” In Int. J. Appl. Earth Obs. Geoinformation 116 Elsevier, 2023, pp. 103145 DOI: https://doi.org/10.1016/j.jag.2022.103145
- “Evaluation of automated pipelines for tree and plot metric estimation from TLS data in tropical forest areas” In Ann. bot. 128.6 Oxford University Press US, 2021, pp. 753–766 DOI: https://doi.org/10.1093/aob/mcab051
- “3D Semantic Parsing of Large-Scale Indoor Spaces” In 2016 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 1534–1543 IEEE DOI: https://doi.org/10.1109/cvpr.2016.170
- “ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes” In 2017 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 2432–2443 IEEE DOI: https://doi.org/10.1109/cvpr.2017.261
- “Semantic3d. net: A new large-scale point cloud classification benchmark” In arXiv prepr. arXiv:1704.03847, 2017 DOI: https://doi.org/10.5194/isprs-annals-iv-1-w1-91-2017
- “Individual rubber tree segmentation based on ground-based LiDAR data and faster R-CNN of deep learning” In Forests 10.9 MDPI, 2019, pp. 793 DOI: https://doi.org/10.3390/f10090793
- “Forest tree detection and segmentation using high resolution airborne LiDAR” In 2019 IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2019, pp. 3898–3904 IEEE DOI: https://doi.org/10.1109/iros40897.2019.8967885
- “Detection, segmentation, and model fitting of individual tree stems from airborne laser scanning of forests using deep learning” In Remote Sensing 12.9 MDPI, 2020, pp. 1469 DOI: https://doi.org/10.3390/rs12091469
- “A Two-stage Approach for Individual Tree Segmentation from TLS Point Clouds” In IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 15 IEEE, 2022, pp. 8682–8693 DOI: https://doi.org/10.1109/jstars.2022.3212445
- “Individual tree crown segmentation directly from UAV-borne LiDAR data using the PointNet of deep learning” In Forests 12.2 MDPI, 2021, pp. 131 DOI: https://doi.org/10.3390/f12020131
- “Forest structural complexity tool—an open source, fully-automated tool for measuring Forest point clouds” In Remote Sensing 13.22 MDPI, 2021, pp. 4677 DOI: https://doi.org/10.3390/rs13224677
- “TLS2trees: a scalable tree segmentation pipeline for TLS data” In bioRxiv Cold Spring Harbor Laboratory, 2022, pp. 2022–12 DOI: https://doi.org/10.1101/2022.12.07.518693
- “Point2Tree (P2T)–framework for parameter tuning of semantic and instance segmentation used with mobile laser scanning data in coniferous forest” In arXiv prepr. arXiv:2305.02651, 2023 DOI: https://doi.org/10.3390/rs15153737
- “Individual tree extraction from urban mobile laser scanning point clouds using deep pointwise direction embedding” In ISPRS J. Photogramm. Remote Sens. 175 Elsevier, 2021, pp. 326–339 DOI: https://doi.org/10.1016/j.isprsjprs.2021.03.002
- “Segmentation of individual trees in urban MLS point clouds using a deep learning framework based on cylindrical convolution network” In Int. J. Appl. Earth Obs. Geoinformation 123 Elsevier, 2023, pp. 103473 DOI: https://doi.org/10.1016/j.jag.2023.103473
- “Towards accurate instance segmentation in large-scale LiDAR point clouds” In arXiv preprint arXiv:2307.02877, 2023
- “SoftGroup for 3D Instance Segmentation on Point Clouds” In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2022, pp. 2708–2717 DOI: https://doi.org/10.1109/cvpr52688.2022.00273
- “Pointgroup: Dual-set point grouping for 3d instance segmentation” In Proc. IEEE/CVF conf. comput. vis. Pattern recognit., 2020, pp. 4867–4876 DOI: https://doi.org/10.1109/cvpr42600.2020.00492
- “Superpoint Transformer for 3D Scene Instance Segmentation” In arXiv prepr. arXiv:2211.15766, 2022 DOI: https://doi.org/10.1609/aaai.v37i2.25335
- “Mask3D for 3D Semantic Instance Segmentation” In arXiv prepr. arXiv:2210.03105, 2022 DOI: https://doi.org/10.1109/icra48891.2023.10160590
- “3D Instances as 1D Kernels” In Comput. Vision–ECCV 2022: 17th Eur. Conf. Tel Aviv, Isr. Oct. 23–27, 2022, Proceedings, Part XXIX, 2022, pp. 235–252 Springer DOI: https://doi.org/10.1007/978-3-031-19818-2“˙14
- Tuan Duc Ngo, Binh-Son Hua and Khoi Nguyen “ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolution” In arXiv prepr. arXiv:2303.00246, 2023 DOI: https://doi.org/10.1109/cvpr52729.2023.01302
- K Calders “Terrestrial Laser Scans—Riegl VZ400, Individual Tree Point Clouds and Cylinder Models, Rushworth Forest. Version 1. Terrestrial Ecosystem Research Network. (Dataset).”, 2014 DOI: https://doi.org/10.4227/05/542B766D5D00D
- “Individual tree point clouds and tree measurements from multi-platform laser scanning in German forests” In Earth Syst. Sci. Data 14.7 Copernicus GmbH, 2022, pp. 2989–3012 DOI: https://doi.org/10.5194/essd-14-2989-2022
- “Automatic tree crown segmentation using dense forest point clouds from Personal Laser Scanning (PLS)” In Int. J. Appl. Earth Obs. Geoinformation 114 Elsevier, 2022, pp. 103025 DOI: https://doi.org/10.1016/j.jag.2022.103025
- “FOR-instance: a UAV laser scanning benchmark dataset for semantic and instance segmentation of individual trees” In arXiv prepr. arXiv:2309.01279, 2023 DOI: https://doi.org/10.1109/ccdc52312.2021.9602282
- “Simulation of silvicultural treatments based on real 3D forest data from mobile laser scanning point clouds” In Trees, For. People Elsevier, 2023, pp. 100372 DOI: https://doi.org/10.1016/j.tfp.2023.100372
- GeoSLAM Ltd “GeoSLAM HUB Version 6”, 2020 URL: https://geoslam.com
- GreenValley International “LIDAR360 Point Cloud Post-Processing Software” Accessed: 26.10.2022, 2022 URL: https://greenvalleyintl.com/LiDAR360
- Daniel Girardeau-Montaut “CloudCompare”, 2022 URL: https://www.cloudcompare.org/
- Olaf Ronneberger, Philipp Fischer and Thomas Brox “U-net: Convolutional networks for biomedical image segmentation” In Int. Conf. Med. Image Comput. Computassist. Interv., 2015, pp. 234–241 Springer DOI: https://doi.org/10.1007/978-3-319-24574-4“˙28
- Timo Hackel, Jan D Wegner and Konrad Schindler “Contour detection in unstructured 3D point clouds” In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1610–1618
- Qian-Yi Zhou, Jaesik Park and Vladlen Koltun “Open3D: A Modern Library for 3D Data Processing”, 2018 DOI: 10.48550/ARXIV.1801.09847
- “Deep hough voting for 3d object detection in point clouds” In proc. IEEE/CVF Int. Conf. Comput. Vis., 2019, pp. 9277–9286 DOI: https://doi.org/10.1109/iccv.2019.00937
- Benjamin Graham, Martin Engelcke and Laurens Van Der Maaten “3d semantic segmentation with submanifold sparse convolutional networks” In Proc. IEEE conf. comput. vis. pattern recognit., 2018, pp. 9224–9232 DOI: https://doi.org/10.1109/cvpr.2018.00961
- “Deep residual learning for image recognition” In Proc. IEEE conf. comput. vis. pattern recognit., 2016, pp. 770–778 DOI: https://doi.org/10.1109/cvpr.2016.90
- Spconv Contributors “Spconv: Spatially Sparse Convolution Library”, https://github.com/traveller59/spconv, 2022
- “A density-based algorithm for discovering clusters in large spatial databases with noise.” In kdd 96, 1996, pp. 226–231 DOI: https://doi.org/10.5120/739-1038
- “Decoupled weight decay regularization”, 2017 DOI: 10.48550/ARXIV.1711.05101
- “Sgdr: Stochastic gradient descent with warm restarts”, 2016 DOI: 10.48550/ARXIV.1608.03983
- “Hierarchical aggregation for 3d instance segmentation” In Proc. IEEE/CVF Int. Conf. Comput. Vis., 2021, pp. 15467–15476 DOI: https://doi.org/10.1109/iccv48922.2021.01518
- Harold W Kuhn “The Hungarian method for the assignment problem” In Nav. res. logist. q. 2.1-2 Wiley Online Library, 1955, pp. 83–97 DOI: https://doi.org/10.1002/nav.20053
- “Realistic forest stand reconstruction from terrestrial LiDAR for radiative transfer modelling” In Remote Sensing 10.6 MDPI, 2018, pp. 933 DOI: https://doi.org/10.3390/rs10060933