Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering (2404.08363v3)

Published 12 Apr 2024 in cs.CV

Abstract: We study the problem of self-supervised 3D scene flow estimation from real large-scale raw point cloud sequences, which is crucial to various tasks like trajectory prediction or instance segmentation. In the absence of ground truth scene flow labels, contemporary approaches concentrate on deducing optimizing flow across sequential pairs of point clouds by incorporating structure based regularization on flow and object rigidity. The rigid objects are estimated by a variety of 3D spatial clustering methods. While state-of-the-art methods successfully capture overall scene motion using the Neural Prior structure, they encounter challenges in discerning multi-object motions. We identified the structural constraints and the use of large and strict rigid clusters as the main pitfall of the current approaches and we propose a novel clustering approach that allows for combination of overlapping soft clusters as well as non-overlapping rigid clusters representation. Flow is then jointly estimated with progressively growing non-overlapping rigid clusters together with fixed size overlapping soft clusters. We evaluate our method on multiple datasets with LiDAR point clouds, demonstrating the superior performance over the self-supervised baselines reaching new state of the art results. Our method especially excels in resolving flow in complicated dynamic scenes with multiple independently moving objects close to each other which includes pedestrians, cyclists and other vulnerable road users. Our codes are publicly available on https://github.com/ctu-vras/let-it-flow.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. SLIM: Self-supervised lidar scene flow and motion segmentation. In ICCV, pages 13126–13136, 2021.
  2. PointFlowNet: Learning representations for rigid motion estimation from point clouds. In CVPR, pages 7962–7971, 2019.
  3. Argoverse: 3D tracking and forecasting with rich maps. In CVPR, pages 8748–8757, 2019.
  4. Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data. IEEE RA-L, 6:6529–6536, 2022.
  5. SC22{}^{\mbox{2}}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT-PCR: A second order spatial compatibility for efficient and robust point cloud registration. In CVPR, pages 13221–13231, 2022.
  6. Bi-PointFlowNet: Bidirectional learning for point cloud based scene flow estimation. In ECCV, 2022.
  7. Re-evaluating lidar scene flow. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6005–6015, January 2024.
  8. 3D object detection with a self-supervised lidar scene flow backbone. In ECCV, 2022.
  9. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, page 226–231. AAAI Press, 1996.
  10. Fast3d: Flow-aware self-training for 3d object detectors. In BMVC, 2021.
  11. Weakly supervised learning of rigid 3D scene flow. In CVPR, pages 5692–5703, June 2021.
  12. Hplflownet: Hierarchical permutohedral lattice flownet for scene flow estimation on large-scale point clouds. In CVPR, pages 3254–3263, 2019.
  13. FlowNet: A deep learning framework for clustering and selection of streamlines and stream surfaces. IEEE Trans. Vis. Comput. Graph., 26(4):1732–1744, 2020.
  14. Moving object detection by 3d flow field analysis. IEEE T-ITS, 22(4):1950–1963, 2021.
  15. Scalable scene flow from point clouds in the real world. IEEE RA-L, 7(2):1589–1596, 2022.
  16. Flowstep3d: Model unrolling for self-supervised scene flow estimation. In CVPR, pages 4112–4121, 2021.
  17. SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow. In CVPR, pages 5281–5290, 2023.
  18. Self-point-flow: Self-supervised scene flow estimation from point clouds with optimal transport and random walk. In CVPR, pages 15577–15586, 2021.
  19. RigidFlow: Self-supervised scene flow learning on point clouds by local rigidity prior. In CVPR, pages 16959–16968, 2022.
  20. Neural scene flow prior. NeurIPS, 34:7838–7851, 2021.
  21. Fast neural scene flow. In ICCV, pages 9878–9890, 2023.
  22. Learning lane graph representations for motion forecasting. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, ECCV, pages 541–556, Cham, 2020. Springer International Publishing.
  23. Toward better boundary preserved supervoxel segmentation for 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 143:39–47, 2018.
  24. FlowNet3D: Learning scene flow in 3d point clouds. CVPR, pages 529–537, 2019.
  25. MeteorNet: Deep learning on dynamic 3d point cloud sequences. In ICCV, 2019.
  26. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, pages 4040–4048, 2016.
  27. Object scene flow for autonomous vehicles. In CVPR, pages 3061–3070, 2015.
  28. Just go with the flow: Self-supervised scene flow estimation. In CVPR, pages 11177–11185, June 2020.
  29. Motion inspired unsupervised perception and prediction in autonomous driving. In ECCV, pages 424–443, 2022.
  30. Scene flow from point clouds with or without learning. In 3DV, pages 261–270, 2020.
  31. FLOT: Scene Flow on Point Clouds Guided by Optimal Transport. In ECCV, 2020.
  32. Self-supervised 3D scene flow estimation guided by superpoints. In CVPR, pages 5271–5280, 2023.
  33. Ziyang Song and Bo Yang. OGC: Unsupervised 3d object segmentation from rigid dynamics of point clouds. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, NeurIPS, 2022.
  34. Scalability in perception for autonomous driving: Waymo open dataset. In CVPR, pages 2446–2454, June 2020.
  35. ZeroFlow: Scalable Scene Flow via Distillation. arXiv:2305.10424, 2023.
  36. Multi-body neural scene flow. In 2024 International Conference on 3D Vision (3DV). IEEE, 2024.
  37. KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way. IEEE Robotics and Automation Letters (RA-L), 8(2):1029–1036, 2023.
  38. Deep parametric continuous convolutional neural networks. In CVPR, pages 2598–2597, June 2018.
  39. IHNet: iterative hierarchical network guided by high-resolution estimated information for scene flow estimation. In ICCV, pages 10073–10082, October 2023.
  40. Argoverse 2: Next generation datasets for self-driving perception and forecasting. In NeurIPS Datasets and Benchmarks, 2021.
  41. PointPWC-Net: Cost volume on point clouds for (self-) supervised scene flow estimation. In ECCV, pages 88–107, 2020.
  42. Deep part induction from articulated object pairs. ACM Trans. Graph., 37(6), dec 2018.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com