Papers
Topics
Authors
Recent
Search
2000 character limit reached

Future Predictive Success-or-Failure Classification for Long-Horizon Robotic Tasks

Published 4 Apr 2024 in cs.RO and cs.CV | (2404.03415v1)

Abstract: Automating long-horizon tasks with a robotic arm has been a central research topic in robotics. Optimization-based action planning is an efficient approach for creating an action plan to complete a given task. Construction of a reliable planning method requires a design process of conditions, e.g., to avoid collision between objects. The design process, however, has two critical issues: 1) iterative trials--the design process is time-consuming due to the trial-and-error process of modifying conditions, and 2) manual redesign--it is difficult to cover all the necessary conditions manually. To tackle these issues, this paper proposes a future-predictive success-or-failure-classification method to obtain conditions automatically. The key idea behind the proposed method is an end-to-end approach for determining whether the action plan can complete a given task instead of manually redesigning the conditions. The proposed method uses a long-horizon future-prediction method to enable success-or-failure classification without the execution of an action plan. This paper also proposes a regularization term called transition consistency regularization to provide easy-to-predict feature distribution. The regularization term improves future prediction and classification performance. The effectiveness of our method is demonstrated through classification and robotic-manipulation experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. L. P. Kaelbling and T. Lozano-Perez, “Hierarchical task and motion planning in the now,” International Conference on Robotics and Automation, pp. 1470–1477, 2011.
  2. M. Janner, Y. Du, J. Tenenbaum, and S. Levine, “Planning with Diffusion for Flexible Behavior Synthesis,” in International Conference on Machine Learning, 2022.
  3. D. Shah, P. Xu, Y. Lu, T. Xiao, A. Toshev, S. Levine, and B. Ichter, “Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning,” no. arXiv:2111.03189, Mar. 2022.
  4. D. Driess, J.-S. Ha, and M. Toussaint, “Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image,” in Robotics: Science and Systems, 2020.
  5. V. N. Hartmann, O. S. Oguz, D. Driess, M. Toussaint, and A. Menges, “Robust Task and Motion Planning for Long-Horizon Architectural Construction Planning,” International Conference on Intelligent Robots and Systems, pp. 6886–6893, 2020.
  6. R. Takano, H. Oyama, and M. Yamakita, “Continuous optimization-based task and motion planning with signal temporal logic specifications for sequential manipulation,” in International Conference on Robotics and Automation, 2021, pp. 8409–8415.
  7. D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson, “Learning Latent Dynamics for Planning from Pixels,” in International Conference on Machine Learning, vol. 97, 2019, pp. 2555–2565.
  8. A. Inceoglu, E. E. Aksoy, A. C. Ak, and S. Sariel, “Fino-net: A deep multimodal sensor fusion framework for manipulation failure detection,” in International Conference on Intelligent Robots and Systems, 2021, pp. 6841–6847.
  9. B. Moldovan, P. Moreno, M. van Otterlo, J. Santos-Victor, and L. De Raedt, “Learning Relational Affordance Models for Robots in Multi-Object Manipulation Tasks,” in International Conference on Robotics and Automation, 2012, pp. 4373–4378.
  10. D. Altan and S. Sariel, “What Went Wrong? Identification of Everyday Object Manipulation Anomalies,” Intelligent Service Robotics, vol. 14, no. 2, pp. 215–234, 2021.
  11. J. Zhang, M. Li, and C. Yang, “Robotic Grasp Detection Using Effective Graspable Feature Selection and Precise Classification,” in International Joint Conference on Neural Networks, 2020, pp. 1–6.
  12. A. Zeng, S. Song, K.-T. Yu, E. Donlon, F. R. Hogan, M. Bauza, D. Ma, O. Taylor, M. Liu, E. Romo et al., “Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching,” The International Journal of Robotics Research, vol. 41, no. 7, pp. 690–705, 2022.
  13. I. Lenz, H. Lee, and A. Saxena, “Deep Learning for Detecting Robotic Grasps,” The International Journal of Robotics Research, vol. 34, no. 4-5, pp. 705–724, 2015.
  14. A. Mousavian, C. Eppner, and D. Fox, “6-DOF GraspNet: Variational Grasp Generation for Object Manipulation,” in International Conference on Computer Vision, 2019, pp. 2901–2910.
  15. S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection,” The International journal of robotics research, vol. 37, no. 4-5, pp. 421–436, 2018.
  16. P. Pastor, M. Kalakrishnan, S. Chitta, E. Theodorou, and S. Schaal, “Skill Learning and Task Outcome Prediction for Manipulation,” in International Conference on Robotics and Automation, 2011, pp. 3828–3834.
  17. A. Lämmle, M. Goes, and P. Tenbrock, “Learning-based Success Validation for Robotic Assembly Tasks,” in International Conference on Emerging Technologies and Factory Automation (ETFA), 2022, pp. 1–4.
  18. A. S. Chen, S. Nair, and C. Finn, “Learning Generalizable Robotic Reward Functions from" in-the-Wild" Human Videos,” in Robotics: Science and Systems, 2021.
  19. D. Furuta, K. Kutsuzawa, S. Sakaino, and T. Tsuji, “Motion Planning With Success Judgement Model Based on Learning From Demonstration,” IEEE Access, vol. 8, pp. 73 142–73 150, 2020.
  20. X. Zhang, Y. Zhu, Y. Ding, Y. Zhu, P. Stone, and S. Zhang, “Visually Grounded Task and Motion Planning for Mobile Manipulation,” in International Conference on Robotics and Automation, 2022, pp. 1925–1931.
  21. R. Y. Rubinstein, “Optimization of computer simulation models with rare events,” European Journal of Operational Research, vol. 99, no. 1, pp. 89–112, 1997.
  22. D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi, “Dream to Control: Learning Behaviors by Latent Imagination,” in International Conference on Learning Representations, 2020.
  23. D. Hafner, T. P. Lillicrap, M. Norouzi, and J. Ba, “Mastering Atari with Discrete World Models,” in International Conference on Learning Representations, 2020.
  24. D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” in International Conference on Learning Representations, 2014.
  25. B. Amos, L. Dinh, S. Cabi, T. Rothörl, S. G. Colmenarejo, A. Muldal, T. Erez, Y. Tassa, N. de Freitas, and M. Denil, “Learning Awareness Models,” in International Conference on Learning Representations, 2018.
  26. A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox, “MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations,” in Conference on Robot Learning, 2023.
  27. J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555, 2014.
  28. A. S. Chen, S. Nair, and C. Finn, “Learning generalizable robotic reward functions from" in-the-wild" human videos,” in Robotics: Science and Systems, 2021.
  29. E. Todorov, T. Erez, and Y. Tassa, “Mujoco: A Physics Engine for Model-Based Control,” in International Conference on Intelligent Robots and Systems, 2012, pp. 5026–5033.
  30. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Computer Vision and Pattern Recognition, 2016, pp. 770–778.
  31. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in Computer Vision and Pattern Recognition, 2009, pp. 248–255.
  32. D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” CoRR, vol. abs/1412.6980, 2014.
  33. T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
  34. L. van der Maaten and G. Hinton, “Visualizing Data using t-SNE,” Journal of Machine Learning Research, vol. 9, no. 86, pp. 2579–2605, 2008.

Summary

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.

Collections

Sign up for free to add this paper to one or more collections.