Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network (2403.18195v3)

Published 27 Mar 2024 in cs.RO and cs.AI

Abstract: Autonomous assembly in robotics and 3D vision presents significant challenges, particularly in ensuring assembly correctness. Presently, predominant methods such as MEPNet focus on assembling components based on manually provided images. However, these approaches often fall short in achieving satisfactory results for tasks requiring long-term planning. Concurrently, we observe that integrating a self-correction module can partially alleviate such issues. Motivated by this concern, we introduce the Single-Step Assembly Error Correction Task, which involves identifying and rectifying misassembled components. To support research in this area, we present the LEGO Error Correction Assembly Dataset (LEGO-ECA), comprising manual images for assembly steps and instances of assembly failures. Additionally, we propose the Self-Correct Assembly Network (SCANet), a novel method to address this task. SCANet treats assembled components as queries, determining their correctness in manual images and providing corrections when necessary. Finally, we utilize SCANet to correct the assembly results of MEPNet. Experimental results demonstrate that SCANet can identify and correct MEPNet's misassembled results, significantly improving the correctness of assembly. Our code and dataset could be found at https://scanet-iros2024.github.io/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. A. S. Micilotta, E.-J. Ong, and R. Bowden, “Detection and tracking of humans by probabilistic body part assembly.” in BMVC, 2005.
  2. J. Anderson, J. E. Dueber, M. Leguia, G. C. Wu, J. A. Goler, A. P. Arkin, and J. D. Keasling, “Bglbricks: A flexible standard for biological part assembly,” J. Med. Biol. Eng., vol. 4, no. 1, 2010.
  3. C.-H. Shen, H. Fu, K. Chen, and S.-M. Hu, “Structure recovery by part assembly,” TOG, vol. 31, no. 6, 2012.
  4. X. Xie, K. Xu, N. J. Mitra, D. Cohen-Or, W. Gong, Q. Su, and B. Chen, “Sketch-to-design: Context-based part assembly,” in Comput Graph Forum, 2013.
  5. P. S. Ogun, Z. Usman, K. Dharmaraj, and M. R. Jackson, “3d vision assisted flexible robotic assembly of machine components,” in ICMV, 2015.
  6. J. S. Laursen, U. P. Schultz, and L.-P. Ellekilde, “Automatic error recovery in robot assembly operations using reverse execution,” in IROS, 2015.
  7. T. Hodaň, J. Matas, and Š. Obdržálek, “On evaluation of 6d object pose estimation,” in ECCV, 2016.
  8. M. Rad and V. Lepetit, “Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth,” in ICCV, 2017.
  9. X. Zhou and Q. Pham, “Can robots assemble an IKEA chair?” Sci. Robotics, vol. 3, no. 17, 2018.
  10. Y. Li, G. Wang, X. Ji, Y. Xiang, and D. Fox, “Deepim: Deep iterative matching for 6d pose estimation,” in ECCV, 2018.
  11. Y. Wang, X. Tan, Y. Yang, X. Liu, E. Ding, F. Zhou, and L. S. Davis, “3d pose estimation for fine-grained object categories,” in ECCV, 2018.
  12. T. Hodan, F. Michel, E. Brachmann, W. Kehl, A. GlentBuch, D. Kraft, B. Drost, J. Vidal, S. Ihrke, X. Zabulis, et al., “Bop: Benchmark for 6d object pose estimation,” in ECCV, 2018.
  13. C. Li, J. Bai, and G. D. Hager, “A unified framework for multi-view multi-class object pose estimation,” in ECCV, 2018.
  14. M. Kocabas, S. Karagoz, and E. Akbas, “Multiposenet: Fast multi-person pose estimation using pose residual network,” in ECCV, 2018.
  15. M. Oberweger, M. Rad, and V. Lepetit, “Making deep heatmaps robust to partial occlusions for 3d object pose estimation,” in ECCV, 2018.
  16. S. Peng, Y. Liu, Q. Huang, X. Zhou, and H. Bao, “Pvnet: Pixel-wise voting network for 6dof pose estimation,” in CVPR, 2019.
  17. Z. Chen, K. Srinet, C. R. Qi, H. Fan, J. Ma, L. Zitnick, D. Guo, T. Xiao, S. Xie, X. Chen, A. Szlam, S. Tulsiani, H. Yu, and J. Gray, “Order-aware generative modeling using the 3d-craft dataset,” in ICCV, 2019.
  18. K. Mo, S. Zhu, A. X. Chang, L. Yi, S. Tripathi, L. J. Guibas, and H. Su, “Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding,” in CVPR, 2019.
  19. Y. Li, K. Mo, L. Shao, M. Sung, and L. J. Guibas, “Learning 3d part assembly from a single image,” in ECCV, 2020.
  20. M. Sundermeyer, M. Durner, E. Y. Puang, Z.-C. Marton, N. Vaskevicius, K. O. Arras, and R. Triebel, “Multi-path learning for object pose estimation across domains,” in CVPR, 2020.
  21. Y. Labbé, J. Carpentier, M. Aubry, and J. Sivic, “Cosypose: Consistent multi-view multi-object 6d pose estimation,” in ECCV, 2020.
  22. G. Zhan, Q. Fan, K. Mo, L. Shao, B. Chen, L. J. Guibas, and H. Dong, “Generative 3d part assembly via dynamic graph learning,” in NeurIPS, 2020.
  23. B. Jones, D. Hildreth, D. Chen, I. Baran, V. G. Kim, and A. Schulz, “Automate: A dataset and learning approach for automatic mating of cad assemblies,” ACM Trans. Graph, vol. 40, 2021.
  24. P. A. Zachares, M. A. Lee, W. Lian, and J. Bohg, “Interpreting contact interactions to overcome failure in robot assembly tasks,” in ICRA, 2021.
  25. H. Chung, J. Kim, B. Knyazev, J. Lee, G. W. Taylor, J. Park, and M. Cho, “Brick-by-brick: Combinatorial construction with deep reinforcement learning,” in NeurIPS, 2021.
  26. R. Wang, Y. Zhang, J. Mao, R. Zhang, C. Cheng, and J. Wu, “Ikea-manual: Seeing shape assembly step by step,” in NeurIPS, 2022.
  27. R. Wang, Y. Zhang, J. Mao, C. Cheng, and J. Wu, “Translating a visual LEGO manual to a machine-executable plan,” in ECCV, 2022.
  28. Z. Wang, S.-W. Zhang, N. Wang, J.-Y. Xu, and D.-J. Cheng, “A feature-based assembly information modeling method for complex products’ 3d assembly design,” Proc Inst Mech Eng B J Eng Manuf, vol. 237, no. 9, 2023.
  29. C. Zheng, W. Wu, C. Chen, T. Yang, S. Zhu, J. Shen, N. Kehtarnavaz, and M. Shah, “Deep learning-based human pose estimation: A survey,” ACM Comput Surv, vol. 56, no. 1, 2023.
  30. C. Liu, K. Shi, K. Zhou, H. Wang, J. Zhang, and H. Dong, “Rgbgrasp: Image-based object grasping by capturing multiple views during robot arm movement with neural radiance fields,” ICRA, 2024.
  31. R. Wu, C. Tie, Y. Du, Y. Zhao, and H. Dong, “Leveraging se (3) equivariance for learning 3d geometric shape assembly,” in ICCV, 2023.
  32. Y. Li, A. Zeng, and S. Song, “Rearrangement planning for general part assembly,” in CoRL, 2023.
  33. R. Wu, C. Tie, Y. Du, Y. Zhao, and H. Dong, “Leveraging SE(3) equivariance for learning 3d geometric shape assembly,” in ICCV, 2023.
  34. J. Corsetti, D. Boscaini, and F. Poiesi, “Revisiting fully convolutional geometric features for object 6d pose estimation,” in ICCV, 2023.
  35. H. Zheng, R. Lee, and Y. Lu, “Ha-vid: A human assembly video dataset for comprehensive assembly knowledge understanding,” ISO4, vol. 36, 2024.
  36. J. Guan, Y. Hao, Q. Wu, S. Li, and Y. Fang, “A survey of 6dof object pose estimation methods for different application scenarios,” Sensors, 2024.
  37. Y. Chen, H. Li, D. Turpin, A. Jacobson, and A. Garg, “Neural shape mating: Self-supervised object assembly with adversarial shape priors,” in CVPR, 2022, pp. 12 714–12 723.
  38. X. Liu, J. Zhang, R. Hu, H. Huang, H. Wang, and L. Yi, “Self-supervised category-level articulated object pose estimation with part-level SE(3) equivariance,” in ICLR, 2023.
  39. A. N. Harish, R. Nagar, and S. Raman, “RGL-NET: A recurrent graph learning framework for progressive part assembly,” in WACV, 2022, pp. 647–656.
  40. Y. Lee, E. S. Hu, and J. J. Lim, “IKEA furniture assembly environment for long-horizon complex manipulation tasks,” in ICRA, 2021, pp. 6343–6349.
  41. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV, 2020, pp. 213–229.
  42. A. Newell, K. Yang, and J. Deng, “Stacked hourglass networks for human pose estimation,” in ECCV, 2016, pp. 483–499.
  43. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016, pp. 770–778.
  44. Y. Li, S. Yang, P. Liu, S. Zhang, Y. Wang, Z. Wang, W. Yang, and S.-T. Xia, “Simcc: A simple coordinate classification perspective for human pose estimation,” in ECCV, 2022, pp. 89–106.
  45. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” vol. 30, 2017.
  46. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in ICLR, 2019.
  47. H. Fan, H. Su, and L. J. Guibas, “A point set generation network for 3d object reconstruction from a single image,” in CVPR, 2017, pp. 2463–2471.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets