Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unsupervised semantic segmentation of urban high-density multispectral point clouds (2410.18520v1)

Published 24 Oct 2024 in cs.CV

Abstract: The availability of highly accurate urban airborne laser scanning (ALS) data will increase rapidly in the future, especially as acquisition costs decrease, for example through the use of drones. Current challenges in data processing are related to the limited spectral information and low point density of most ALS datasets. Another challenge will be the growing need for annotated training data, frequently produced by manual processes, to enable semantic interpretation of point clouds. This study proposes to semantically segment new high-density (1200 points per square metre on average) multispectral ALS data with an unsupervised ground-aware deep clustering method GroupSP inspired by the unsupervised GrowSP algorithm. GroupSP divides the scene into superpoints as a preprocessing step. The neural network is trained iteratively by grouping the superpoints and using the grouping assignments as pseudo-labels. The predictions for the unseen data are given by over-segmenting the test set and mapping the predicted classes into ground truth classes manually or with automated majority voting. GroupSP obtained an overall accuracy (oAcc) of 97% and a mean intersection over union (mIoU) of 80%. When compared to other unsupervised semantic segmentation methods, GroupSP outperformed GrowSP and non-deep K-means. However, a supervised random forest classifier outperformed GroupSP. The labelling efforts in GroupSP can be minimal; it was shown, that the GroupSP can semantically segment seven urban classes (building, high vegetation, low vegetation, asphalt, rock, football field, and gravel) with oAcc of 95% and mIoU of 75% using only 0.004% of the available annotated points in the mapping assignment. Finally, the multispectral information was examined; adding each new spectral channel improved the mIoU. Additionally, echo deviation was valuable, especially when distinguishing ground-level classes.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (92)
  1. The ASTER spectral library version 2.0. Remote Sensing of Environment, 113(4):711–715, 2009. doi:https://doi.org/10.1016/j.rse.2008.11.007.
  2. Spatial cross-validation for globally distributed data. In International Conference on Discovery Science, pages 127–140, 2022. doi:10.1007/978-3-031-18840-4_10.
  3. T. Blaschke. Object based image analysis for remote sensing. ISPRS Journal of Photogrammetry and Remote Sensing, 65(1):2–16, 2010. doi:10.1016/j.isprsjprs.2009.06.004.
  4. I. S. Bozchalooi and K. Youcef-Toumi. LiDAR device based on scanning mirrors array and multi-frequency laser modulation, U.S. Patent US10649072B2, May. 2020.
  5. L. Breiman. Random forests. Machine Learning, 45:5–32, 2001. doi:10.1023/A:1010933404324.
  6. Self-supervised pre-training boosts semantic scene segmentation on LiDAR data. In 18th International Conference on Machine Vision and Applications, pages 1–6, 2023. doi:10.23919/MVA57639.2023.10216191.
  7. A feature perturbation weakly supervised learning network for airborne multispectral LiDAR pointcloud classification. International Journal of Applied Earth Observation and Geoinformation, 127:103683, 2024. doi:10.1016/j.jag.2024.103683.
  8. Two-channel hyperspectral LiDAR with a supercontinuum laser source. Sensors, 10(7):7057–7066, 2010. doi:10.3390/s100707057.
  9. PointDC: Unsupervised semantic segmentation of 3D point clouds via cross-modal distillation and super-voxel clustering. In IEEE/CVF International Conference on Computer Vision, pages 14244–14253, 2023. doi:10.1109/ICCV51070.2023.01314.
  10. 4D spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3070–3079, 2019. doi:10.1109/CVPR.2019.00319.
  11. DC3DCD: Unsupervised learning for multiclass 3D point cloud change detection. ISPRS Journal of Photogrammetry and Remote Sensing, 206:168–183, 2023a. doi:10.1016/j.isprsjprs.2023.10.022.
  12. Deep unsupervised learning for 3D ALS point clouds change detection. ISPRS Open Journal of Photogrammetry and Remote Sensing, 9:100044, 2023b. doi:10.1016/j.ophoto.2023.100044.
  13. Dimensionality based scale selection in 3D LiDAR point clouds. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXVIII-5/W12:97–102, 2012. doi:10.5194/isprsarchives-XXXVIII-5-W12-97-2011.
  14. Classification of airborne multispectral LiDAR point clouds for land cover mapping. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11(6):2068–2078, 2018. doi:10.1109/JSTARS.2018.2835483.
  15. Identifying conifer tree vs. deciduous shrub and tree regeneration trajectories in a space-for-time boreal peatland fire chronosequence using multispectral LiDAR. Atmosphere, 13(1):112, 2022. doi:10.3390/atmos13010112.
  16. National Land Survey of Finland. Finnish national laser scanning data, 2024. URL https://www.maanmittauslaitos.fi/en/maps-and-spatial-data/datasets-and-interfaces/product-descriptions/laser-scanning-data-5-p. Accessed 12 September 2024.
  17. 3D semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9224–9232, 2018.
  18. S. Guinard and L. Landrieu. Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-1/W1:151–157, 2017. doi:10.5194/isprs-archives-XLII-1-W1-151-2017.
  19. MCTNet: Multiscale cross-attention-based transformer network for semantic segmentation of large-scale point cloud. IEEE Transactions on Geoscience and Remote Sensing, 61:1–20, 2023. doi:10.1109/TGRS.2023.3322579.
  20. Full waveform hyperspectral LiDAR for terrestrial laser scanning. Opt. Express, 20(7):7119–7127, 2012. doi:10.1364/OE.20.007119.
  21. Individual tree segmentation and species classification using high-density close-range multispectral laser scanning data. ISPRS Open Journal of Photogrammetry and Remote Sensing, 9:100039, 2023. doi:10.1016/j.ophoto.2023.100039.
  22. D. S. Hall. Color LiDAR scanner, U.S. Patent US8675181B2, Mar. 2014.
  23. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026–1034, 2015. doi:10.1109/ICCV.2015.123.
  24. OFFS-Net: Optimal feature fusion-based spectral information network for airborne point cloud classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16:141–152, 2023. doi:10.1109/JSTARS.2022.3223698.
  25. Beyond supervised learning in remote sensing: A systematic review of deep learning approaches. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17:1035–1052, 2024. doi:10.1109/JSTARS.2023.3316733.
  26. A spectral analysis of 25 boreal tree species. Silva Fennica, 51(4):7753, 2017. doi:10.14214/sf.7753.
  27. RandLA-Net: Efficient semantic segmentation of large-scale point clouds. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11105–11114, 2020. doi:10.1109/CVPR42600.2020.01112.
  28. GraNet: Global relation-aware attentional network for semantic segmentation of ALS point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 177:1–20, 2021. doi:10.1016/j.isprsjprs.2021.04.017.
  29. A simple framework of few-shot learning using sparse annotations for semantic segmentation of 3D point clouds. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17:5147–5158, 2024. doi:10.1109/JSTARS.2024.3363243.
  30. Multispectral LiDAR point cloud classification using SE-PointNet++. Remote Sensing, 13(13):2516, 2021. doi:10.3390/rs13132516.
  31. S. Kaasalainen. Multispectral terrestrial LiDAR: State of the art and challenges. Laser Scanning, pages 5–18, 2019.
  32. S. Kaasalainen and T. Malkamäki. Hyperspectral LiDAR: A progress report. Optics and Photonics News, 32(11):38–43, 2021. doi:10.1364/OPN.32.11.000038.
  33. Feasibility of multispectral airborne laser scanning data for road mapping. IEEE Geoscience and Remote Sensing Letters, 14:294–298, 2017. doi:10.1109/LGRS.2016.2631261.
  34. Automatic labelling for semantic segmentation of VHR satellite images: Application of airborne laser scanner data and object-based image analysis. ISPRS Open Journal of Photogrammetry and Remote Sensing, 9:100046, 2023. doi:10.1016/j.ophoto.2023.100046.
  35. Modern trends in hyperspectral image analysis: A review. IEEE Access, 6:14118–14129, 2018. doi:10.1109/ACCESS.2018.2812999.
  36. Usgs spectral library version 7 data: Us geological survey data release. United States Geological Survey (USGS): Reston, VA, USA, 61, 2017. doi:10.3133/ds1035.
  37. AGFP-Net: Attentive geometric feature pyramid network for land cover classification using airborne multispectral LiDAR data. International Journal of Applied Earth Observation and Geoinformation, 108:102723, 2022. doi:10.1016/j.jag.2022.102723.
  38. Active and incremental learning for semantic ALS point cloud segmentation. ISPRS Journal of Photogrammetry and Remote Sensing, 169:73–92, 2020. doi:/10.1016/j.isprsjprs.2020.09.003.
  39. Local and global encoder network for semantic segmentation of airborne laser scanning point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 176:151–168, 2021. doi:10.1016/j.isprsjprs.2021.04.016.
  40. Weakly supervised semantic segmentation of airborne laser scanning point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 187:79–100, 2022. doi:10.1016/j.isprsjprs.2022.03.001.
  41. Classification of tree species classes in a hemi-boreal forest from multispectral airborne laser scanning data using a mini raster cell method. International Journal of Applied Earth Observation and Geoinformation, 100:102334, 2021. doi:http10.1016/j.jag.2021.102334.
  42. Context-aware network for semantic segmentation toward large-scale point clouds in urban environments. IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2022. doi:10.1109/TGRS.2022.3182776.
  43. U3DS3: Unsupervised 3D semantic scene segmentation. In IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3747–3756, 2024. doi:10.1109/WACV57701.2024.00372.
  44. S. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129–137, 1982. doi:10.1109/TIT.1982.1056489.
  45. I. Loshchilov and F. Hutter. Preprint: Decoupled weight decay regularization, 2019. doi:10.48550/arXiv.1711.05101
  46. Portable hyperspectral lidar utilizing 5 GHz multichannel full waveform digitization. Opt. Express, 27(8):A468–A480, 2019. doi:10.1364/OE.27.00A468.
  47. Object-based analysis of multispectral airborne laser scanner data for land cover classification and map updating. ISPRS Journal of Photogrammetry and Remote Sensing, 128:298–313, 2017. doi:10.1016/j.isprsjprs.2017.04.005.
  48. Toward utilizing multitemporal multispectral airborne laser scanning, sentinel-2, and mobile laser scanning in map updating. Journal of Applied Remote Sensing, 13:044504, 2019. doi:10.1117/1.JRS.13.4.044504.
  49. Assessing forest structural and physiological information content of multi-spectral LiDAR waveforms by radiative transfer modelling. Remote Sensing of Environment, 113(10):2152–2163, 2009. doi:10.1016/j.rse.2009.05.019.
  50. Contextual classification of lidar data and building object detection in urban areas. ISPRS Journal of Photogrammetry and Remote Sensing, 87:152–165, 2014. doi:10.1016/j.isprsjprs.2013.11.001.
  51. Land-cover classification of multispectral LiDAR data using CNN with optimized hyper-parameters. ISPRS Journal of Photogrammetry and Remote Sensing, 166:241–254, 2020. doi:h10.1016/j.isprsjprs.2020.05.022.
  52. PDAL contributors. PDAL: The point data abstraction library, 2024. URL https://pdal.io/. Accessed 8 October 2024.
  53. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  54. Automatic region-growing system for the segmentation of large point clouds. Automation in Construction, 138:104250, 2022. doi:10.1016/j.autcon.2022.104250.
  55. PointNet: Deep learning on point sets for 3D classification and segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, pages 77–85, 2017a. doi:10.1109/CVPR.2017.16.
  56. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems, 2017b.
  57. Airborne small-footprint full-waveform LiDAR data for urban land cover classification. Frontiers in Environmental Science, 10, 2022. doi:10.3389/fenvs.2022.972960.
  58. Efficient 3D semantic segmentation with superpoint transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17195–17204, 2023.
  59. J. R. Schott. Remote sensing: The image chain approach. Oxford University Press on Demand, 2007.
  60. Spectral imaging for remote sensing. Lincoln laboratory journal, 14(1):3–28, 2003.
  61. L. N. Smith and N. Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, page 1100612, 2019. doi:10.1117/12.2520589.
  62. Z. Song and B. Yang. OGC: Unsupervised 3D object segmentation from rigid dynamics of point clouds. Advances in Neural Information Processing Systems, 35:30798–30812, 2022.
  63. Analysis of land cover classification using multi-wavelength LiDAR system. Applied Sciences, 7(7):663, 2017. doi:10.3390/app7070663.
  64. The map service of the City of Espoo. Ortophoto (c) Helsinki, Espoo, Vantaa, Kauniainen, Kirkkonummi, Kerava, Nurmijärvi, HSY, HSL and The Finnish Defence Forces 2023. URL https://kartat.espoo.fi/IMS/en/Map. Accessed 12 September 2024
  65. E. van Rees. The first multispectral airborne lidar sensor. GeoInformatics, 18(1):10–12, 2015.
  66. P. S. Viswanathan and B. Xue. Scanning 3D imaging device with power control using multiple wavelengths, U.S. Patent US20190212447A1, Dec. 2020.
  67. Airborne dual-wavelength lidar data for classifying land cover. Remote Sensing, 6(1):700–715, 2014. doi:10.3390/rs6010700.
  68. Semantic segmentation of urban airborne LiDAR point clouds based on fusion attention mechanism and multi-scale features. Remote Sensing, 15(21):5248, 2023. doi:10.3390/rs15215248.
  69. P. Wang and W. Yao. A new weakly supervised approach for ALS point cloud semantic segmentation. ISPRS Journal of Photogrammetry and Remote Sensing, 188:237–254, 2022. doi:10.1016/j.isprsjprs.2022.04.016.
  70. Multi-attribute smooth graph convolutional network for multispectral points classification. Science China Technological Sciences, 64(11):2509–2522, 2021. doi:h10.1007/s11431-020-1871-8.
  71. Dynamic Graph CNN for Learning on Point Clouds. ACM Transactions on Graphics, 38(5):146, 2019. doi:10.1145/3326362.
  72. Multi-spectral laser scanning for inspection of building surfaces: State of the art and future concepts. In Proceedings of the 7th International Conference on Virtual Reality, Archaeology and Intelligent Cultural Heritage, pages 147–154, 2006. doi:htt10.2312/VAST/VAST06/147-154.
  73. Directionally constrained fully convolutional neural network for airborne LiDAR point cloud classification. ISPRS Journal of Photogrammetry and Remote Sensing, 162:50–62, 2020. doi:h10.1016/j.isprsjprs.2020.02.004.
  74. Airborne LiDAR point cloud classification with global-local graph attention convolution neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 173:181–194, 2021. doi:10.1016/j.isprsjprs.2021.01.007.
  75. Semantic segmentation on LiDAR point cloud in urban area using deep learning. In International Workshop on Big Data and Information Security, pages 63–66, 2019. doi:10.1109/IWBIS.2019.8935882.
  76. Evaluating the potential of multispectral airborne LiDAR for topographic mapping and land cover classification. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, II-3/W5:113–119, 2015. doi:10.5194/isprsannals-II-3-W5-113-2015.
  77. Unsupervised point cloud representation learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):11321–11339, 2023. doi:10.1109/TPAMI.2023.3262786.
  78. Multiscale adjacency matrix CNN: Learning on multispectral LiDAR point cloud via multiscale local graph convolution. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17:855–870, 2024. doi:10.1109/JSTARS.2023.3335300.
  79. Single-sensor solution to tree species classification using multispectral airborne laser scanning. Remote Sensing, 9(2):108, 2017. doi:10.3390/rs9020108.
  80. A hybrid capsule network for land cover classification using multispectral LiDAR data. IEEE Geoscience and Remote Sensing Letters, 17(7):1263–1267, 2020. doi:10.1109/LGRS.2019.2940505.
  81. CapViT: Cross-context capsule vision transformers for land cover classification with airborne multispectral LiDAR data. International Journal of Applied Earth Observation and Geoinformation, 111:102837, 2022. doi:https://doi.org/10.1016/j.jag.2022.102837.
  82. Multilevel context feature fusion for semantic segmentation of ALS point cloud. IEEE Geoscience and Remote Sensing Letters, 20:1–5, 2023a. doi:10.1109/LGRS.2023.3294246.
  83. Recurrent residual dual attention network for airborne laser scanning point cloud semantic segmentation. IEEE Transactions on Geoscience and Remote Sensing, 61:1–14, 2023b. doi:10.1109/TGRS.2023.3285207.
  84. Unsupervised learning of ALS point clouds for 3D terrain scene clustering. IEEE Geoscience and Remote Sensing Letters, 19:1–5, 2022a. doi:10.1109/LGRS.2020.3047096.
  85. Deep-learning-based point cloud semantic segmentation: A survey. Electronics, 12(17):3642, 2023a. doi:10.3390/electronics12173642.
  86. An easy-to-use airborne LiDAR data filtering method based on cloth simulation. Remote Sensing, 8(6):501, 2016. doi:10.3390/rs8060501.
  87. PointBoost: LiDAR-enhanced semantic segmentation of remote sensing imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16:5618–5628, 2023b. doi:10.1109/JSTARS.2023.3286912.
  88. HAVANA: Hard negative sample-aware self-supervised contrastive learning for airborne laser scanning point cloud semantic segmentation. Remote Sensing, 16(3):485, 2024. doi:10.3390/rs16030485.
  89. Introducing improved transformer to land cover classification using multispectral LiDAR point clouds. Remote Sensing, 14(15):3808, 2022b. doi:10.3390/rs14153808.
  90. GrowSP: Unsupervised semantic segmentation of 3D point clouds. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17619–17629, 2023c. doi:10.1109/CVPR52729.2023.01690.
  91. ALS point cloud classification with small training data set based on transfer learning. IEEE Geoscience and Remote Sensing Letters, 17(8):1406–1410, 2020. doi:10.1109/LGRS.2019.2947608.
  92. Airborne multispectral LiDAR point cloud classification with a feature reasoning-based graph convolution network. International Journal of Applied Earth Observation and Geoinformation, 105:102634, 2021. doi:10.1016/j.jag.2021.102634.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets