KG-Planner: Knowledge-Informed Graph Neural Planning for Collaborative Manipulators (2405.07962v1)
Abstract: This paper presents a novel knowledge-informed graph neural planner (KG-Planner) to address the challenge of efficiently planning collision-free motions for robots in high-dimensional spaces, considering both static and dynamic environments involving humans. Unlike traditional motion planners that struggle with finding a balance between efficiency and optimality, the KG-Planner takes a different approach. Instead of relying solely on a neural network or imitating the motions of an oracle planner, our KG-Planner integrates explicit physical knowledge from the workspace. The integration of knowledge has two key aspects: (1) we present an approach to design a graph that can comprehensively model the workspace's compositional structure. The designed graph explicitly incorporates critical elements such as robot joints, obstacles, and their interconnections. This representation allows us to capture the intricate relationships between these elements. (2) We train a Graph Neural Network (GNN) that excels at generating nearly optimal robot motions. In particular, the GNN employs a layer-wise propagation rule to facilitate the exchange and update of information among workspace elements based on their connections. This propagation emphasizes the influence of these elements throughout the planning process. To validate the efficacy and efficiency of our KG-Planner, we conduct extensive experiments in both static and dynamic environments. These experiments include scenarios with and without human workers. The results of our approach are compared against existing methods, showcasing the superior performance of the KG-Planner. A short video introduction of this work is available (video link provided in the paper).
- D. Berenson, P. Abbeel, and K. Goldberg, “A robot path planning framework that learns from experience,” in 2012 IEEE International Conference on Robotics and Automation. IEEE, 2012, pp. 3671–3678.
- M.-L. Lee, X. Liang, B. Hu, G. Onel, S. Behdad, and M. Zheng, “A review of prospects and opportunities in disassembly with human–robot collaboration,” Journal of Manufacturing Science and Engineering, vol. 146, no. 2, 2024.
- M.-L. Lee, W. Liu, S. Behdad, X. Liang, and M. Zheng, “Robot-assisted disassembly sequence planning with real-time human motion prediction,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 53, no. 1, pp. 438–450, 2022.
- M.-L. Lee, S. Behdad, X. Liang, and M. Zheng, “Task allocation and planning for product disassembly with human–robot collaboration,” Robotics and Computer-Integrated Manufacturing, vol. 76, p. 102306, 2022.
- C.-M. Huang and B. Mutlu, “Anticipatory robot control for efficient human-robot collaboration,” in 2016 11th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, 2016, pp. 83–90.
- M. Mohanan and A. Salgoankar, “A survey of robotic motion planning in dynamic environments,” Robotics and Autonomous Systems, vol. 100, pp. 171–185, 2018.
- M. Rickert, O. Brock, and A. Knoll, “Balancing exploration and exploitation in motion planning,” in 2008 IEEE International Conference on Robotics and Automation. IEEE, 2008, pp. 2812–2817.
- A. H. Qureshi and Y. Ayaz, “Intelligent bidirectional rapidly-exploring random trees for optimal motion planning in complex cluttered environments,” Robotics and Autonomous Systems, vol. 68, pp. 1–11, 2015.
- Qureshi, Ahmed Hussain and Ayaz, Yasar, “Potential functions based sampling heuristic for optimal path planning,” Autonomous Robots, vol. 40, pp. 1079–1093, 2016.
- W. Liu, X. Liang, and M. Zheng, “Task-constrained motion planning considering uncertainty-informed human motion prediction for human–robot collaborative disassembly,” IEEE/ASME Transactions on Mechatronics, 2023.
- M. Pfeiffer, M. Schaeuble, J. Nieto, R. Siegwart, and C. Cadena, “From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots,” in 2017 ieee international conference on robotics and automation (icra). IEEE, 2017, pp. 1527–1533.
- M. Hamandi, M. D’Arcy, and P. Fazli, “Deepmotion: Learning to navigate like humans,” in 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2019, pp. 1–7.
- B. Ichter and M. Pavone, “Robot motion planning in learned latent spaces,” IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2407–2414, 2019.
- A. Khan, A. Ribeiro, V. Kumar, and A. G. Francis, “Graph neural networks for motion planning,” arXiv preprint arXiv:2006.06248, 2020.
- R. Zhang, C. Yu, J. Chen, C. Fan, and S. Gao, “Learning-based motion planning in dynamic environments using gnns and temporal encoding,” Advances in Neural Information Processing Systems, vol. 35, pp. 30 003–30 015, 2022.
- E. W. Dijkstra, “A note on two problems in connexion with graphs,” in Edsger Wybe Dijkstra: His Life, Work, and Legacy, 2022, pp. 287–290.
- P. E. Hart, N. J. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” IEEE transactions on Systems Science and Cybernetics, vol. 4, no. 2, pp. 100–107, 1968.
- S. Koenig, M. Likhachev, and D. Furcy, “Lifelong planning a*,” Artificial Intelligence, vol. 155, no. 1-2, pp. 93–146, 2004.
- D. Dolgov, S. Thrun, M. Montemerlo, and J. Diebel, “Practical search techniques in path planning for autonomous driving,” Ann Arbor, vol. 1001, no. 48105, pp. 18–80, 2008.
- M. Likhachev, D. I. Ferguson, G. J. Gordon, A. Stentz, and S. Thrun, “Anytime dynamic a*: An anytime, replanning algorithm.” in ICAPS, vol. 5, 2005, pp. 262–271.
- L. E. Kavraki, P. Svestka, J.-C. Latombe, and M. H. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,” IEEE transactions on Robotics and Automation, vol. 12, no. 4, pp. 566–580, 1996.
- S. M. LaValle, “Rapidly-exploring random trees: A new tool for path planning,” Research Report, 1998.
- S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” The international journal of robotics research, vol. 30, no. 7, pp. 846–894, 2011.
- J. D. Gammell, S. S. Srinivasa, and T. D. Barfoot, “Informed rrt: Optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic,” in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014, pp. 2997–3004.
- Gammell, Jonathan D and Srinivasa, Siddhartha S and Barfoot, Timothy D, “Batch informed trees (bit*): Sampling-based optimal planning via the heuristically guided search of implicit random geometric graphs,” in 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 2015, pp. 3067–3074.
- L. Janson, E. Schmerling, A. Clark, and M. Pavone, “Fast marching tree: A fast marching sampling-based method for optimal motion planning in many dimensions,” The International journal of robotics research, vol. 34, no. 7, pp. 883–921, 2015.
- N. Ratliff, M. Zucker, J. A. Bagnell, and S. Srinivasa, “Chomp: Gradient optimization techniques for efficient motion planning,” in 2009 IEEE international conference on robotics and automation. IEEE, 2009, pp. 489–494.
- M. Kalakrishnan, S. Chitta, E. Theodorou, P. Pastor, and S. Schaal, “Stomp: Stochastic trajectory optimization for motion planning,” in 2011 IEEE international conference on robotics and automation. IEEE, 2011, pp. 4569–4574.
- P. Reynoso-Mora, W. Chen, and M. Tomizuka, “A convex relaxation for the time-optimal trajectory planning of robotic manipulators along predetermined geometric paths,” Optimal Control Applications and Methods, vol. 37, no. 6, pp. 1263–1281, 2016.
- H.-C. Lin, C. Liu, and M. Tomizuka, “Fast robot motion planning with collision avoidance and temporal optimization,” in 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV). IEEE, 2018, pp. 29–35.
- T. Marcucci, M. Petersen, D. von Wrangel, and R. Tedrake, “Motion planning around obstacles with convex optimization,” Science Robotics, vol. 8, no. 84, p. eadf7843, 2023.
- S. Zimmermann, G. Hakimifard, M. Zamora, R. Poranne, and S. Coros, “A multi-level optimization framework for simultaneous grasping and motion planning,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2966–2972, 2020.
- J. Wang, T. Zhang, N. Ma, Z. Li, H. Ma, F. Meng, and M. Q.-H. Meng, “A survey of learning-based robot motion planning,” IET Cyber-Systems and Robotics, vol. 3, no. 4, pp. 302–314, 2021.
- J. Wang, W. Chi, C. Li, C. Wang, and M. Q.-H. Meng, “Neural rrt*: Learning-based optimal path planning,” IEEE Transactions on Automation Science and Engineering, vol. 17, no. 4, pp. 1748–1758, 2020.
- P. Lehner and A. Albu-Schäffer, “The repetition roadmap for repetitive constrained motion planning,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3884–3891, 2018.
- C. Liu, W. Liu, Z. Chen, and M. Zheng, “A deep-convlstm collision prediction model for manipulators in dynamic environment,” in The 9th IFAC Symposium on Mechatronic Systems & The 16th International Conference on Motion and Vibration Control, 2022.
- C. Yu and S. Gao, “Reducing collision checking for sampling-based motion planning using graph neural networks,” Advances in Neural Information Processing Systems, vol. 34, pp. 4274–4289, 2021.
- J. Huh, V. Isler, and D. D. Lee, “Cost-to-go function generating networks for high dimensional motion planning,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 8480–8486.
- M. J. Bency, A. H. Qureshi, and M. C. Yip, “Neural path planning: Fixed time, near-optimal path generation via oracle imitation,” in 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2019, pp. 3965–3972.
- A. H. Qureshi, Y. Miao, A. Simeonov, and M. C. Yip, “Motion planning networks: Bridging the gap between learning-based and classical motion planners,” IEEE Transactions on Robotics, vol. 37, no. 1, pp. 48–66, 2020.
- L. Li, Y. Miao, A. H. Qureshi, and M. C. Yip, “Mpc-mpnet: Model-predictive motion planning networks for fast, near-optimal planning under kinodynamic constraints,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4496–4503, 2021.
- Y. Zhang, H. Wang, Y. Luo, L. Yu, H. Hu, H. Shan, and T. Q. Quek, “Three-dimensional convolutional neural network pruning with regularization-based method,” in 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019, pp. 4270–4274.
- Y. Chen, H. Fan, B. Xu, Z. Yan, Y. Kalantidis, M. Rohrbach, S. Yan, and J. Feng, “Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 3435–3444.
- Y. Liu, D. Jiang, H. Duan, Y. Sun, G. Li, B. Tao, J. Yun, Y. Liu, and B. Chen, “Dynamic gesture recognition algorithm based on 3d convolutional neural network,” Computational Intelligence and Neuroscience, vol. 2021, 2021.
- Q. Li, G. Chalvatzaki, J. Peters, and Y. Wang, “Directed acyclic graph neural network for human motion prediction,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 3197–3204.
- W. Liu, X. Liang, and M. Zheng, “Dynamic model informed human motion prediction based on unscented kalman filter,” IEEE/ASME Transactions on Mechatronics, vol. 27, no. 6, pp. 5287–5295, 2022.
- S. H. Park, B. Kim, C. M. Kang, C. C. Chung, and J. W. Choi, “Sequence-to-sequence prediction of vehicle trajectory via lstm encoder-decoder architecture,” in 2018 IEEE intelligent vehicles symposium (IV). IEEE, 2018, pp. 1672–1678.
- B. Sanchez-Lengeling, E. Reif, A. Pearce, and A. B. Wiltschko, “A gentle introduction to graph neural networks,” Distill, 2021, https://distill.pub/2021/gnn-intro.
- M. Ding, Y. Liu, C. Yang, and X. Lan, “Visual manipulation relationship detection based on gated graph neural network for robotic grasping,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022, pp. 1404–1410.
- Y. Lin, A. S. Wang, E. Undersander, and A. Rai, “Efficient and interpretable robot manipulation with graph neural networks,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2740–2747, 2022.
- D.-A. Huang, S. Nair, D. Xu, Y. Zhu, A. Garg, L. Fei-Fei, S. Savarese, and J. C. Niebles, “Neural task graphs: Generalizing to unseen tasks from a single video demonstration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8565–8574.
- F. Gama, J. Bruna, and A. Ribeiro, “Stability properties of graph neural networks,” IEEE Transactions on Signal Processing, vol. 68, pp. 5680–5695, 2020.
- T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.