Papers
Topics
Authors
Recent
2000 character limit reached

Has Anything Changed? 3D Change Detection by 2D Segmentation Masks (2312.01148v1)

Published 2 Dec 2023 in cs.CV

Abstract: As capturing devices become common, 3D scans of interior spaces are acquired on a daily basis. Through scene comparison over time, information about objects in the scene and their changes is inferred. This information is important for robots and AR and VR devices, in order to operate in an immersive virtual experience. We thus propose an unsupervised object discovery method that identifies added, moved, or removed objects without any prior knowledge of what objects exist in the scene. We model this problem as a combination of a 3D change detection and a 2D segmentation task. Our algorithm leverages generic 2D segmentation masks to refine an initial but incomplete set of 3D change detections. The initial changes, acquired through render-and-compare likely correspond to movable objects. The incomplete detections are refined through graph optimization, distilling the information of the 2D segmentation masks in the 3D space. Experiments on the 3Rscan dataset prove that our method outperforms competitive baselines, with SoTA results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Objects can move: 3d change detection by geometric transformation consistency. In Eur. Conf. Comput. Vis., pages 108–124. Springer, 2022.
  2. Meta-rooms: Building and maintaining long term spatial models in a dynamic world. In 2014 IEEE/RSJ international conference on intelligent robots and systems, pages 1854–1861. IEEE, 2014.
  3. Unsupervised object segmentation through change detection in a long term autonomy scenario. In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages 1181–1187. IEEE, 2016.
  4. Discovering objects that can move. In IEEE Conf. Comput. Vis. Pattern Recog., pages 11789–11798, 2022.
  5. Object discovery from motion-guided tokens. In IEEE Conf. Comput. Vis. Pattern Recog., pages 22972–22981, 2023.
  6. Jonathan T Barron. A generalization of otsu’s method and minimum error thresholding. In Eur. Conf. Comput. Vis., pages 455–470. Springer, 2020.
  7. Contrastive lift: 3d object instance segmentation by slow-fast contrastive fusion. arXiv preprint arXiv:2306.04633, 2023.
  8. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
  9. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124–1137, 2004.
  10. Omni3d: A large benchmark and model for 3d object detection in the wild. In IEEE Conf. Comput. Vis. Pattern Recog., pages 13154–13164, 2023.
  11. Abo: Dataset and benchmarks for real-world 3d object understanding. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21126–21136, 2022.
  12. Objaverse-xl: A universe of 10m+ 3d objects. arXiv preprint arXiv:2307.05663, 2023.
  13. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
  14. Tsdf-based change detection for consistent long-term dense reconstruction and dynamic object discovery. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 5237–5244. IEEE, 2017.
  15. Toward lifelong object segmentation from change detection in dense rgb-d maps. In 2013 European Conference on Mobile Robots, pages 178–185. IEEE, 2013.
  16. Efficient 2d and 3d facade segmentation using auto-context. IEEE Trans. Pattern Anal. Mach. Intell., 40(5):1273–1280, 2017.
  17. Real-time rgb-d camera relocalization. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 173–179. IEEE, 2013.
  18. Rescan: Inductive instance segmentation for indoor rgbd scans. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2541–2550, 2019.
  19. Mask r-cnn. In Int. Conf. Comput. Vis., pages 2961–2969, 2017.
  20. Toward object discovery and modeling via 3-d scene comparison. In 2011 IEEE International Conference on Robotics and Automation (ICRA), pages 2623–2629. IEEE, 2011.
  21. Toward online 3-d object segmentation and mapping. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 3193–3200. IEEE, 2014.
  22. Spatial change detection using normal distributions transform. ROBOMECH Journal, 6(1):1–13, 2019.
  23. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
  24. On information and sufficiency. The annals of mathematical statistics, 22(1):79–86, 1951.
  25. Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs. SIAM Journal on Imaging Sciences, 10(4):1724–1766, 2017.
  26. A structured regularization framework for spatially smoothing semantic labelings of 3d point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 132:102–118, 2017.
  27. On-the-fly detection of novel objects in indoor environments. In 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 900–907. IEEE, 2017.
  28. Robust and efficient object change detection by combining global semantic information and local geometric verification. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8453–8460. IEEE, 2020.
  29. Microsoft coco: Common objects in context. In Eur. Conf. Comput. Vis., pages 740–755. Springer, 2014.
  30. An object-based semantic world model for long-term change detection and semantic querying. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3851–3858. IEEE, 2012.
  31. Change detection in 3d models based on camera images. In 9th Workshop on Planning, Perception and Navigation for Intelligent Vehicles at the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2017.
  32. Fast image-based geometric change detection given a 3d model. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 6308–6315. IEEE, 2018.
  33. Voxel cloud connectivity segmentation-supervoxels for point clouds. In IEEE Conf. Comput. Vis. Pattern Recog., pages 2027–2034, 2013.
  34. Changesim: Towards end-to-end online scene change detection in industrial indoor environments. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8578–8585. IEEE, 2021.
  35. Deep hough voting for 3d object detection in point clouds. In Int. Conf. Comput. Vis., 2019.
  36. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10901–10911, 2021.
  37. Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects. In 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 10–20. IEEE, 2018.
  38. The change you want to see (now in 3d). In Int. Conf. Comput. Vis., pages 2060–2069, 2023.
  39. Nothing stands still: A spatiotemporal benchmark on 3d point cloud registration under large geometric and temporal change. arXiv preprint arXiv:2311.09346, 2023.
  40. Image based detection of geometric changes in urban environments. In Int. Conf. Comput. Vis., pages 2336–2343. IEEE, 2011.
  41. City-scale change detection in cadastral 3d models using images. In IEEE Conf. Comput. Vis. Pattern Recog., pages 113–120, 2013.
  42. Image-based 4-d reconstruction using 3-d change detection. In Eur. Conf. Comput. Vis., pages 31–45. Springer, 2014.
  43. Rio: 3d object instance re-localization in changing indoor environments. In IEEE Conf. Comput. Vis. Pattern Recog., pages 7658–7667, 2019.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up