Papers
Topics
Authors
Recent
2000 character limit reached

Toward Holistic Planning and Control Optimization for Dual-Arm Rearrangement (2404.06758v1)

Published 10 Apr 2024 in cs.RO

Abstract: Long-horizon task and motion planning (TAMP) is notoriously difficult to solve, let alone optimally, due to the tight coupling between the interleaved (discrete) task and (continuous) motion planning phases, where each phase on its own is frequently an NP-hard or even PSPACE-hard computational challenge. In this study, we tackle the even more challenging goal of jointly optimizing task and motion plans for a real dual-arm system in which the two arms operate in close vicinity to solve highly constrained tabletop multi-object rearrangement problems. Toward that, we construct a tightly integrated planning and control optimization pipeline, Makespan-Optimized Dual-Arm Planner (MODAP) that combines novel sampling techniques for task planning with state-of-the-art trajectory optimization techniques. Compared to previous state-of-the-art, MODAP produces task and motion plans that better coordinate a dual-arm system, delivering significantly improved execution time improvements while simultaneously ensuring that the resulting time-parameterized trajectory conforms to specified acceleration and jerk limits.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. S. D. Han, N. M. Stiffler, A. Krontiris, K. E. Bekris, and J. Yu, “Complexity results and fast methods for optimal tabletop rearrangement with overhand grasps,” The International Journal of Robotics Research, vol. 37, no. 13-14, pp. 1775–1795, 2018.
  2. K. Gao, S. W. Feng, B. Huang, and J. Yu, “Minimizing running buffers for tabletop object rearrangement: Complexity, fast algorithms, and applications,” The International Journal of Robotics Research, vol. 42, no. 10, pp. 755–776, 2023. [Online]. Available: https://doi.org/10.1177/02783649231178565
  3. J. E. Hopcroft, J. T. Schwartz, and M. Sharir, “On the complexity of motion planning for multiple independent objects; pspace-hardness of the ‘warehouseman’s problem’,” The International Journal of Robotics Research, vol. 3, no. 4, pp. 76–88, 1984.
  4. X. Zhang, Y. Zhu, Y. Ding, Y. Zhu, P. Stone, and S. Zhang, “Visually grounded task and motion planning for mobile manipulation,” in 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 1925–1931.
  5. L. Wang, X. Meng, Y. Xiang, and D. Fox, “Hierarchical policies for cluttered-scene grasping with latent plans,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2883–2890, 2022.
  6. B. Kim, L. Shimanuki, L. P. Kaelbling, and T. Lozano-Pérez, “Representation, learning, and planning algorithms for geometric task and motion planning,” The International Journal of Robotics Research, vol. 41, no. 2, pp. 210–231, 2022. [Online]. Available: https://doi.org/10.1177/02783649211038280
  7. K. Gao and J. Yu, “Toward efficient task planning for dual-arm tabletop object rearrangement,” in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 10 425–10 431.
  8. B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V. Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos, N. Ratliff, and D. Fox, “Curobo: Parallelized collision-free robot motion generation,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 8112–8119.
  9. C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Pérez, “Integrated task and motion planning,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 4, no. 1, pp. 265–293, 2021. [Online]. Available: https://doi.org/10.1146/annurev-control-091420-084139
  10. M. Toussaint, “Logic-geometric programming: An optimization-based approach to combined task and motion planning.” in IJCAI, 2015, pp. 1930–1936.
  11. N. T. Dantam, Z. K. Kingston, S. Chaudhuri, and L. E. Kavraki, “Incremental task and motion planning: A constraint-based approach.” in Robotics: Science and systems, vol. 12.   Ann Arbor, MI, USA, 2016, p. 00052.
  12. ——, “An incremental constraint-based framework for task and motion planning,” The International Journal of Robotics Research, vol. 37, no. 10, pp. 1134–1151, 2018.
  13. L. P. Kaelbling and T. Lozano-Pérez, “Hierarchical task and motion planning in the now,” in 2011 IEEE International Conference on Robotics and Automation, 2011, pp. 1470–1477.
  14. J. Luo, C. Xu, X. Geng, G. Feng, K. Fang, L. Tan, S. Schaal, and S. Levine, “Multistage cable routing through hierarchical imitation learning,” IEEE Transactions on Robotics, vol. 40, pp. 1476–1491, 2024.
  15. A. M. Wells, N. T. Dantam, A. Shrivastava, and L. E. Kavraki, “Learning feasibility for task and motion planning in tabletop environments,” IEEE robotics and automation letters, vol. 4, no. 2, pp. 1255–1262, 2019.
  16. C. R. Garrett, T. Lozano-Pérez, and L. P. Kaelbling, “Pddlstream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning,” in Proceedings of the international conference on automated planning and scheduling, vol. 30, 2020, pp. 440–448.
  17. Z. Zhao, W. S. Lee, and D. Hsu, “Large language models as commonsense knowledge for large-scale task planning,” in Advances in Neural Information Processing Systems, A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, Eds., vol. 36.   Curran Associates, Inc., 2023, pp. 31 967–31 987.
  18. M. Stilman and J. J. Kuffner, “Navigation among movable obstacles: Real-time reasoning in complex environments,” International Journal of Humanoid Robotics, vol. 2, no. 04, pp. 479–503, 2005.
  19. M. Stilman and J. Kuffner, “Planning among movable obstacles with artificial constraints,” The International Journal of Robotics Research, vol. 27, no. 11-12, pp. 1295–1307, 2008.
  20. K. Okada, A. Haneda, H. Nakai, M. Inaba, and H. Inoue, “Environment manipulation planner for humanoid robots using task graph that generates action sequence,” in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566), vol. 2.   IEEE, 2004, pp. 1174–1179.
  21. M. Levihn, J. Scholz, and M. Stilman, “Hierarchical decision theoretic planning for navigation among movable obstacles,” in Algorithmic Foundations of Robotics X.   Springer, 2013, pp. 19–35.
  22. K. Gao, D. Lau, B. Huang, K. E. Bekris, and J. Yu, “Fast high-quality tabletop rearrangement in bounded workspace,” arXiv preprint arXiv:2110.12325, 2021.
  23. A. Cosgun, T. Hermans, V. Emeli, and M. Stilman, “Push planning for object placement on cluttered table surfaces,” in 2011 IEEE/RSJ international conference on intelligent robots and systems.   IEEE, 2011, pp. 4627–4632.
  24. R. Wang, K. Gao, D. Nakhimovich, J. Yu, and K. E. Bekris, “Uniform object rearrangement: From complete monotone primitives to efficient non-monotone informed search,” in IEEE International Conference on Robotics and Automation, 2021.
  25. R. Wang, Y. Miao, and K. E. Bekris, “Efficient and high-quality prehensile rearrangement in cluttered and confined spaces,” arXiv preprint arXiv:2110.02814, 2021.
  26. C. Nam, J. Lee, Y. Cho, J. Lee, D. H. Kim, and C. Kim, “Planning for target retrieval using a robotic manipulator in cluttered and occluded environments,” arXiv preprint arXiv:1907.03956, 2019.
  27. H. Zhang, Y. Lu, C. Yu, D. Hsu, X. Lan, and N. Zheng, “INVIGORATE: Interactive Visual Grounding and Grasping in Clutter,” in Proceedings of Robotics: Science and Systems, Virtual, July 2021.
  28. E. R. Vieira, D. Nakhimovich, K. Gao, R. Wang, J. Yu, and K. E. Bekris, “Persistent homology for effective non-prehensile manipulation,” arXiv preprint arXiv:2202.02937, 2022.
  29. B. Huang, T. Guo, A. Boularias, and J. Yu, “Self-supervised monte carlo tree search learning for object retrieval in clutter,” arXiv preprint arXiv:2202.01426, 2022.
  30. B. Huang, S. D. Han, A. Boularias, and J. Yu, “Dipn: Deep interaction prediction network with application to clutter removal,” in IEEE International Conference on Robotics and Automation, 2021.
  31. J. E. King, V. Ranganeni, and S. S. Srinivasa, “Unobservable monte carlo planning for nonprehensile rearrangement tasks,” in 2017 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2017, pp. 4681–4688.
  32. B. Huang, X. Zhang, and J. Yu, “Toward optimal tabletop rearrangement with multiple manipulation primitives,” 2023.
  33. A. Krontiris and K. E. Bekris, “Dealing with difficult instances of object rearrangement.” in Robotics: Science and Systems, vol. 1123, 2015.
  34. ——, “Efficiently solving general rearrangement tasks: A fast extension primitive for an incremental sampling-based planner,” in 2016 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2016, pp. 3924–3931.
  35. K. Gao, Y. Ding, S. Zhang, and J. Yu, “Orla*: Mobile manipulator-based object rearrangement with lazy a*,” 2023.
  36. S. Bereg and A. Dumitrescu, “The lifting model for reconfiguration,” Discrete & Computational Geometry, vol. 35, no. 4, pp. 653–669, 2006.
  37. S. H. Cheong, B. Y. Cho, J. Lee, C. Kim, and C. Nam, “Where to relocate?: Object rearrangement inside cluttered and confined environments for robotic manipulation,” in 2020 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2020, pp. 7791–7797.
  38. K. Gao and J. Yu, “On the utility of buffers in pick-n-swap based lattice rearrangement,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 5786–5792.
  39. S. Karaman and E. Frazzoli, “Sampling-based algorithms for optimal motion planning,” The international journal of robotics research, vol. 30, no. 7, pp. 846–894, 2011.
  40. L. Janson, B. Ichter, and M. Pavone, “Deterministic sampling-based motion planning: Optimality, complexity, and performance,” The International Journal of Robotics Research, vol. 37, no. 1, pp. 46–61, 2018.
  41. B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018, pp. 7087–7094.
  42. J. D. Gammell, S. S. Srinivasa, and T. D. Barfoot, “Informed rrt*: Optimal incremental path planning focused through an admissible ellipsoidal heuristic,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), vol. 2, no. 1, 2014, pp. 3–1.
  43. S. LAVALLE, “Rapidly-exploring random trees : a new tool for path planning,” Research Report 9811, 1998. [Online]. Available: https://cir.nii.ac.jp/crid/1573950399665672960
  44. L. Kavraki, P. Svestka, J.-C. Latombe, and M. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,” IEEE Transactions on Robotics and Automation, vol. 12, no. 4, pp. 566–580, 1996.
  45. J. Schulman, Y. Duan, J. Ho, A. Lee, I. Awwal, H. Bradlow, J. Pan, S. Patil, K. Goldberg, and P. Abbeel, “Motion planning with sequential convex optimization and convex collision checking,” The International Journal of Robotics Research, vol. 33, no. 9, pp. 1251–1270, 2014.
  46. M. Zucker, N. Ratliff, A. D. Dragan, M. Pivtoraiko, M. Klingensmith, C. M. Dellin, J. A. Bagnell, and S. S. Srinivasa, “Chomp: Covariant hamiltonian optimization for motion planning,” The International journal of robotics research, vol. 32, no. 9-10, pp. 1164–1193, 2013.
  47. T. Marcucci, M. Petersen, D. von Wrangel, and R. Tedrake, “Motion planning around obstacles with convex optimization,” Science robotics, vol. 8, no. 84, p. eadf7843, 2023.
  48. J. Urain, N. Funk, J. Peters, and G. Chalvatzaki, “Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion,” in 2023 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2023, pp. 5923–5930.
  49. D. Malyuta, T. P. Reynolds, M. Szmuk, T. Lew, R. Bonalli, M. Pavone, and B. Açıkmeşe, “Convex optimization for trajectory generation: A tutorial on generating dynamically feasible trajectories reliably and efficiently,” IEEE Control Systems Magazine, vol. 42, no. 5, pp. 40–113, 2022.
  50. Y. Li, Z. Littlefield, and K. E. Bekris, “Asymptotically optimal sampling-based kinodynamic planning,” The International Journal of Robotics Research, vol. 35, no. 5, pp. 528–564, 2016.
  51. K. Hauser and Y. Zhou, “Asymptotically optimal planning by feasible kinodynamic planning in a state–cost space,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1431–1443, 2016.
  52. W. Thomason, Z. Kingston, and L. E. Kavraki, “Motions in microseconds via vectorized sampling-based planning,” 2023.
  53. D. Zhang, C. Liang, X. Gao, K. Wu, and Z. Pan, “Provably robust semi-infinite program under collision constraints via subdivision,” arXiv preprint arXiv:2302.01135, 2023.
  54. B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V. Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos, et al., “curobo: Parallelized collision-free minimum-jerk robot motion generation,” arXiv preprint arXiv:2310.17274, 2023.
  55. H. Pham and Q.-C. Pham, “A new approach to time-optimal path parameterization based on reachability analysis,” IEEE Transactions on Robotics, vol. 34, no. 3, pp. 645–659, 2018.
  56. S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, “Automatic generation and detection of highly reliable fiducial markers under occlusion,” Pattern Recognition, vol. 47, no. 6, pp. 2280–2292, 2014.
  57. A. P. Lindvig, “ur_rtde,” https://gitlab.com/sdurobotics/ur˙rtde, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Slide Deck Streamline Icon: https://streamlinehq.com

Whiteboard

Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up