Fast Ergodic Search with Kernel Functions (2403.01536v1)
Abstract: Ergodic search enables optimal exploration of an information distribution while guaranteeing the asymptotic coverage of the search space. However, current methods typically have exponential computation complexity in the search space dimension and are restricted to Euclidean space. We introduce a computationally efficient ergodic search method. Our contributions are two-fold. First, we develop a kernel-based ergodic metric and generalize it from Euclidean space to Lie groups. We formally prove the proposed metric is consistent with the standard ergodic metric while guaranteeing linear complexity in the search space dimension. Secondly, we derive the first-order optimality condition of the kernel ergodic metric for nonlinear systems, which enables efficient trajectory optimization. Comprehensive numerical benchmarks show that the proposed method is at least two orders of magnitude faster than the state-of-the-art algorithm. Finally, we demonstrate the proposed algorithm with a peg-in-hole insertion task. We formulate the problem as a coverage task in the space of SE(3) and use a 30-second-long human demonstration as the prior distribution for ergodic coverage. Ergodicity guarantees the asymptotic solution of the peg-in-hole problem so long as the solution resides within the prior information distribution, which is seen in the 100\% success rate.
- R. Murphy, “Human-robot interaction in rescue robotics,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 34, no. 2, pp. 138–153, May 2004, conference Name: IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).
- K. Shah, G. Ballard, A. Schmidt, and M. Schwager, “Multidrone aerial surveys of penguin colonies in Antarctica,” Science Robotics, vol. 5, no. 47, p. eabc3000, Oct. 2020, publisher: American Association for the Advancement of Science. [Online]. Available: https://www.science.org/doi/10.1126/scirobotics.abc3000
- I. Abraham and T. D. Murphey, “Decentralized Ergodic Control: Distribution-Driven Sensing and Exploration for Multiagent Systems,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 2987–2994, Oct. 2018, conference Name: IEEE Robotics and Automation Letters.
- S. Shetty, J. Silvério, and S. Calinon, “Ergodic Exploration Using Tensor Train: Applications in Insertion Tasks,” IEEE Transactions on Robotics, vol. 38, no. 2, pp. 906–921, Apr. 2022.
- I. Abraham and T. D. Murphey, “Active Learning of Dynamics for Data-Driven Control Using Koopman Operators,” IEEE Transactions on Robotics, vol. 35, no. 5, pp. 1071–1083, Oct. 2019.
- A. Prabhakar and T. Murphey, “Mechanical intelligence for learning embodied sensor-object relationships,” Nature Communications, vol. 13, no. 1, p. 4108, Jul. 2022, number: 1 Publisher: Nature Publishing Group. [Online]. Available: https://www.nature.com/articles/s41467-022-31795-2
- G. Mathew and I. Mezić, “Metrics for ergodicity and design of ergodic dynamics for multi-agent systems,” Physica D: Nonlinear Phenomena, vol. 240, no. 4-5, pp. 432–442, Feb. 2011. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S016727891000285X
- G. Mathew, I. Mezić, and L. Petzold, “A multiscale measure for mixing,” Physica D: Nonlinear Phenomena, vol. 211, no. 1, pp. 23–46, Nov. 2005. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167278905003313
- C. Chen, T. D. Murphey, and M. A. MacIver, “Tuning movement for sensing in an uncertain world,” eLife, vol. 9, p. e52371, Sep. 2020. [Online]. Available: https://elifesciences.org/articles/52371
- M. Sun, A. Pinosky, I. Abraham, and T. Murphey, “Scale-Invariant Fast Functional Registration,” in Robotics Research, ser. Springer Proceedings in Advanced Robotics. Springer International Publishing, 2022.
- L. M. Miller and T. D. Murphey, “Trajectory optimization for continuous ergodic exploration,” in 2013 American Control Conference, Jun. 2013, pp. 4196–4201, iSSN: 2378-5861.
- ——, “Trajectory optimization for continuous ergodic exploration on the motion group SE(2),” in 52nd IEEE Conference on Decision and Control, Dec. 2013, pp. 4517–4522, iSSN: 0191-2216. [Online]. Available: https://ieeexplore.ieee.org/document/6760585
- I. Abraham, A. Prabhakar, and T. D. Murphey, “An Ergodic Measure for Active Learning From Equilibrium,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 3, pp. 917–931, Jul. 2021. [Online]. Available: https://ieeexplore.ieee.org/document/9312988
- Y. Tang, “A Note on Monte Carlo Integration in High Dimensions,” The American Statistician, vol. 0, no. 0, pp. 1–7, 2023, publisher: Taylor & Francis _eprint: https://doi.org/10.1080/00031305.2023.2267637. [Online]. Available: https://doi.org/10.1080/00031305.2023.2267637
- A. Mavrommati, E. Tzorakoleftherakis, I. Abraham, and T. D. Murphey, “Real-Time Area Coverage and Target Localization Using Receding-Horizon Ergodic Exploration,” IEEE Transactions on Robotics, vol. 34, no. 1, pp. 62–80, Feb. 2018.
- A. Kalinowska, A. Prabhakar, K. Fitzsimons, and T. Murphey, “Ergodic imitation: Learning from what to do and what not to do,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), May 2021, pp. 3648–3654, iSSN: 2577-087X. [Online]. Available: https://ieeexplore.ieee.org/document/9561746
- D. Ehlers, M. Suomalainen, J. Lundell, and V. Kyrki, “Imitating Human Search Strategies for Assembly,” in 2019 International Conference on Robotics and Automation (ICRA), May 2019, pp. 7821–7827, iSSN: 2577-087X. [Online]. Available: https://ieeexplore.ieee.org/document/8793780
- C. Lerch, D. Dong, and I. Abraham, “Safety-Critical Ergodic Exploration in Cluttered Environments via Control Barrier Functions,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), May 2023, pp. 10 205–10 211. [Online]. Available: https://ieeexplore.ieee.org/document/10161032
- Z. Ren, A. K. Srinivasan, B. Vundurthy, I. Abraham, and H. Choset, “A Pareto-Optimal Local Optimization Framework for Multiobjective Ergodic Search,” IEEE Transactions on Robotics, vol. 39, no. 5, pp. 3452–3463, Oct. 2023. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10158414
- D. Dong, H. Berger, and I. Abraham, “Time Optimal Ergodic Search,” in Robotics: Science and Systems XIX. Robotics: Science and Systems Foundation, Jul. 2023. [Online]. Available: http://www.roboticsproceedings.org/rss19/p082.pdf
- L. M. Miller, Y. Silverman, M. A. MacIver, and T. D. Murphey, “Ergodic Exploration of Distributed Information,” IEEE Transactions on Robotics, vol. 32, no. 1, pp. 36–52, Feb. 2016. [Online]. Available: https://ieeexplore.ieee.org/document/7350162
- D. J. Rosenkrantz, R. E. Stearns, and P. M. Lewis, II, “An Analysis of Several Heuristics for the Traveling Salesman Problem,” SIAM Journal on Computing, vol. 6, no. 3, pp. 563–581, Sep. 1977, publisher: Society for Industrial and Applied Mathematics. [Online]. Available: https://epubs.siam.org/doi/abs/10.1137/0206041
- J. Solà, J. Deray, and D. Atchuthan, “A micro Lie theory for state estimation in robotics,” Dec. 2021, arXiv:1812.01537 [cs]. [Online]. Available: http://arxiv.org/abs/1812.01537
- T. Fan and T. Murphey, “Online Feedback Control for Input-Saturated Robotic Systems on Lie Groups,” in Robotics: Science and Systems XII. Robotics: Science and Systems Foundation, 2016. [Online]. Available: http://www.roboticsproceedings.org/rss12/p27.pdf
- Yunfeng Wang and G. Chirikjian, “Error propagation on the Euclidean group with applications to manipulator kinematics,” IEEE Transactions on Robotics, vol. 22, no. 4, pp. 591–602, Aug. 2006. [Online]. Available: http://ieeexplore.ieee.org/document/1668246/
- Y. Wang and G. S. Chirikjian, “Nonparametric Second-order Theory of Error Propagation on Motion Groups,” The International Journal of Robotics Research, vol. 27, no. 11-12, pp. 1258–1273, Nov. 2008, publisher: SAGE Publications Ltd STM. [Online]. Available: https://doi.org/10.1177/0278364908097583
- G. Chirikjian and M. Kobilarov, “Gaussian approximation of non-linear measurement models on Lie groups,” in 53rd IEEE Conference on Decision and Control. Los Angeles, CA, USA: IEEE, Dec. 2014, pp. 6401–6406. [Online]. Available: http://ieeexplore.ieee.org/document/7040393/
- P. Chauchat, A. Barrau, and S. Bonnabel, “Invariant smoothing on Lie Groups,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid: IEEE, Oct. 2018, pp. 1703–1710. [Online]. Available: https://ieeexplore.ieee.org/document/8594068/
- J. G. Mangelson, M. Ghaffari, R. Vasudevan, and R. M. Eustice, “Characterizing the Uncertainty of Jointly Distributed Poses in the Lie Algebra,” IEEE Transactions on Robotics, vol. 36, no. 5, pp. 1371–1388, Oct. 2020. [Online]. Available: https://ieeexplore.ieee.org/document/9124716/
- R. Hartley, M. Ghaffari, R. M. Eustice, and J. W. Grizzle, “Contact-aided invariant extended Kalman filtering for robot state estimation,” The International Journal of Robotics Research, vol. 39, no. 4, pp. 402–430, Mar. 2020, publisher: SAGE Publications Ltd STM. [Online]. Available: https://doi.org/10.1177/0278364919894385
- A. Saccon, J. Hauser, and A. P. Aguiar, “Optimal Control on Lie Groups: The Projection Operator Approach,” IEEE Transactions on Automatic Control, vol. 58, no. 9, pp. 2230–2245, Sep. 2013.
- I. Oseledets, “Tensor-Train Toolbox (ttpy),” Jan. 2024, original-date: 2012-08-21T18:22:27Z. [Online]. Available: https://github.com/oseledets/ttpy
- L. Dagum and R. Menon, “OpenMP: an industry standard API for shared-memory programming,” IEEE Computational Science and Engineering, vol. 5, no. 1, pp. 46–55, Jan. 1998, conference Name: IEEE Computational Science and Engineering. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/660313
- H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard, “Recent Advances in Robot Learning from Demonstration,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 3, no. 1, pp. 297–330, 2020, _eprint: https://doi.org/10.1146/annurev-control-100819-063206. [Online]. Available: https://doi.org/10.1146/annurev-control-100819-063206
- Z. Wu, W. Lian, C. Wang, M. Li, S. Schaal, and M. Tomizuka, “Prim-LAfD: A Framework to Learn and Adapt Primitive-Based Skills from Demonstrations for Insertion Tasks,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 4120–4125, Jan. 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2405896323021572
- K. Zhang, C. Wang, H. Chen, J. Pan, M. Y. Wang, and W. Zhang, “Vision-based Six-Dimensional Peg-in-Hole for Practical Connector Insertion,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). London, United Kingdom: IEEE, May 2023, pp. 1771–1777. [Online]. Available: https://ieeexplore.ieee.org/document/10161116/
- B. Wen, W. Lian, K. Bekris, and S. Schaal, “You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration,” in Robotics: Science and Systems XVIII. Robotics: Science and Systems Foundation, Jun. 2022. [Online]. Available: http://www.roboticsproceedings.org/rss18/p044.pdf
- P. Englert and M. Toussaint, “Learning manipulation skills from a single demonstration,” The International Journal of Robotics Research, vol. 37, no. 1, pp. 137–154, Jan. 2018, publisher: SAGE Publications Ltd STM. [Online]. Available: https://doi.org/10.1177/0278364917743795
- M. Saveriano, F. J. Abu-Dakka, A. Kramberger, and L. Peternel, “Dynamic movement primitives in robotics: A tutorial survey,” The International Journal of Robotics Research, vol. 42, no. 13, pp. 1133–1184, Nov. 2023, publisher: SAGE Publications Ltd STM. [Online]. Available: https://doi.org/10.1177/02783649231201196
- D. K. Jha, D. Romeres, W. Yerazunis, and D. Nikovski, “Imitation and Supervised Learning of Compliance for Robotic Assembly,” in 2022 European Control Conference (ECC). London, United Kingdom: IEEE, Jul. 2022, pp. 1882–1889. [Online]. Available: https://ieeexplore.ieee.org/document/9838102/
- T. Davchev, K. S. Luck, M. Burke, F. Meier, S. Schaal, and S. Ramamoorthy, “Residual Learning From Demonstration: Adapting DMPs for Contact-Rich Manipulation,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 4488–4495, Apr. 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9709544
- J. Luo*, O. Sushkov*, R. Pevceviciute*, W. Lian, C. Su, M. Vecerik, N. Ye, S. Schaal, and J. Scholz, “Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study,” in Robotics: Science and Systems XVII. Robotics: Science and Systems Foundation, Jul. 2021. [Online]. Available: http://www.roboticsproceedings.org/rss17/p088.pdf
- K.-H. Ahn, M. Na, and J.-B. Song, “Robotic assembly strategy via reinforcement learning based on force and visual information,” Robotics and Autonomous Systems, vol. 164, p. 104399, Jun. 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889023000386
- Y. Guo, J. Gao, Z. Wu, C. Shi, and J. Chen, “Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward,” in Proceedings of The 6th Conference on Robot Learning. PMLR, Mar. 2023, pp. 1146–1156, iSSN: 2640-3498. [Online]. Available: https://proceedings.mlr.press/v205/guo23a.html
- “Arm Control System — support.rethinkrobotics.com.” [Online]. Available: https://support.rethinkrobotics.com/support/solutions/articles/80000980284-arm-control-system
- W. Chen, R. Khardon, and L. Liu, “AK: Attentive Kernel for Information Gathering,” in Robotics: Science and Systems XVIII. Robotics: Science and Systems Foundation, Jun. 2022. [Online]. Available: http://www.roboticsproceedings.org/rss18/p047.pdf
- S. Levine and V. Koltun, “Guided Policy Search,” in Proceedings of the 30th International Conference on Machine Learning. PMLR, May 2013, pp. 1–9, iSSN: 1938-7228. [Online]. Available: https://proceedings.mlr.press/v28/levine13.html