Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Constraint Inference in Control Tasks from Expert Demonstrations via Inverse Optimization (2304.03367v3)

Published 6 Apr 2023 in cs.RO

Abstract: Inferring unknown constraints is a challenging and crucial problem in many robotics applications. When only expert demonstrations are available, it becomes essential to infer the unknown domain constraints to deploy additional agents effectively. In this work, we propose an approach to infer affine constraints in control tasks after observing expert demonstrations. We formulate the constraint inference problem as an inverse optimization problem, and we propose an alternating optimization scheme that infers the unknown constraints by minimizing a KKT residual objective. We demonstrate the effectiveness of our method in a number of simulations, and show that our method can infer less conservative constraints than a recent baseline method, while maintaining comparable safety guarantees.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. A. Y. Ng, S. Russell et al., “Algorithms for inverse reinforcement learning.” in Icml, vol. 1, 2000, p. 2.
  2. A. Hussein, M. M. Gaber, E. Elyan, and C. Jayne, “Imitation learning: A survey of learning methods,” ACM Computing Surveys (CSUR), vol. 50, no. 2, pp. 1–35, 2017.
  3. G. Liu, Y. Luo, A. Gaurav, K. Rezaee, and P. Poupart, “Benchmarking constraint inference in inverse reinforcement learning,” arXiv preprint arXiv:2206.09670, 2022.
  4. G. Chou, N. Ozay, and D. Berenson, “Learning constraints from locally-optimal demonstrations under cost function uncertainty,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3682–3690, 2020.
  5. T. C. Chan, R. Mahmood, and I. Y. Zhu, “Inverse optimization: Theory and applications,” arXiv preprint arXiv:2109.03920, 2021.
  6. D. R. Scobee and S. S. Sastry, “Maximum likelihood constraint inference for inverse reinforcement learning,” arXiv preprint arXiv:1909.05477, 2019.
  7. S. Malik, U. Anwar, A. Aghasi, and A. Ahmed, “Inverse constrained reinforcement learning,” in International Conference on Machine Learning.   PMLR, 2021, pp. 7390–7399.
  8. D. Papadimitriou, U. Anwar, and D. S. Brown, “Bayesian methods for constraint inference in reinforcement learning,” Transactions on Machine Learning Research.
  9. C. Pérez-D’Arpino and J. A. Shah, “C-learn: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy,” in 2017 IEEE International Conference on Robotics and Automation (ICRA).   IEEE, 2017, pp. 4058–4065.
  10. G. Chou, D. Berenson, and N. Ozay, “Learning constraints from demonstrations,” in International Workshop on the Algorithmic Foundations of Robotics.   Springer, 2018, pp. 228–245.
  11. A. Robey, H. Hu, L. Lindemann, H. Zhang, D. V. Dimarogonas, S. Tu, and N. Matni, “Learning control barrier functions from expert demonstrations,” in 2020 59th IEEE Conference on Decision and Control (CDC).   IEEE, 2020, pp. 3717–3724.
  12. T. L. Molloy, J. J. Ford, and T. Perez, “Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons,” Automatica, vol. 120, p. 109109, 2020.
  13. A. Agrawal, S. Barratt, and S. Boyd, “Learning convex optimization models,” IEEE/CAA Journal of Automatica Sinica, vol. 8, no. 8, pp. 1355–1364, 2021.
  14. M. Menner, P. Worsnop, and M. N. Zeilinger, “Constrained inverse optimal control with application to a human manipulation task,” IEEE Transactions on Control Systems Technology, vol. 29, no. 2, pp. 826–834, 2019.
  15. J. Li, C.-Y. Chiu, L. Peters, S. Sojoudi, C. Tomlin, and D. Fridovich-Keil, “Cost inference for feedback dynamic games from noisy partial state observations and incomplete trajectories,” arXiv preprint arXiv:2301.01398, 2023.
  16. R. K. Ahuja and J. B. Orlin, “Inverse optimization,” Operations Research, vol. 49, no. 5, pp. 771–783, 2001.
  17. T. C. Chan and N. Kaw, “Inverse optimization for the recovery of constraint parameters,” European Journal of Operational Research, vol. 282, no. 2, pp. 415–427, 2020.
  18. K. Ghobadi and H. Mahmoudzadeh, “Inferring linear feasible regions using inverse optimization,” European Journal of Operational Research, vol. 290, no. 3, pp. 829–843, 2021.
  19. P. Englert, N. A. Vien, and M. Toussaint, “Inverse kkt: Learning cost functions of manipulation tasks from demonstrations,” The International Journal of Robotics Research, vol. 36, no. 13-14, pp. 1474–1488, 2017.
  20. C. Awasthi, “Forward and inverse methods in optimal control and dynamic game theory,” Ph.D. dissertation, University of Minnesota, 2019.
  21. M. Menner and M. N. Zeilinger, “Maximum likelihood methods for inverse learning of optimal controllers,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 5266–5272, 2020.
  22. A. Keshavarz, Y. Wang, and S. Boyd, “Imputing a convex objective function,” in 2011 IEEE international symposium on intelligent control.   IEEE, 2011, pp. 613–619.
  23. J. Gorski, F. Pfeuffer, and K. Klamroth, “Biconvex sets and optimization with biconvex functions: a survey and extensions,” Mathematical methods of operations research, vol. 66, no. 3, pp. 373–407, 2007.
  24. R. de Lazcano, K. Andreas, J. J. Tai, S. R. Lee, and J. Terry, “Gymnasium robotics,” 2023. [Online]. Available: http://github.com/Farama-Foundation/Gymnasium-Robotics
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now