Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Inverse Optimal Control as an Errors-in-Variables Problem (2312.03532v1)

Published 6 Dec 2023 in eess.SY and cs.SY

Abstract: Inverse optimal control (IOC) is about estimating an unknown objective of interest given its optimal control sequence. However, truly optimal demonstrations are often difficult to obtain, e.g., due to human errors or inaccurate measurements. This paper presents an IOC framework for objective estimation from multiple sub-optimal demonstrations in constrained environments. It builds upon the Karush-Kuhn-Tucker optimality conditions, and addresses the Errors-In-Variables problem that emerges from the use of sub-optimal data. The approach presented is applied to various systems in simulation, and consistency guarantees are provided for linear systems with zero mean additive noise, polytopic constraints, and objectives with quadratic features.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (38)
  1. From inverse optimal control to inverse reinforcement learning: A historical review. Annual Reviews in Control, 2020.
  2. Brian D. O. Anderson and John B. Moore. Optimal Filtering. Courier Corporation, January 2005.
  3. Sequential quadratic programming. Acta numerica, 4:1–51, 1995.
  4. Bayesian two-compartment and classic single-compartment minimal models: comparison on insulin modified IVGTT and effect of experiment reduction. IEEE transactions on biomedical engineering, pages 1301–1309, 2003.
  5. Understanding the metropolis-hastings algorithm. The american statistician, 1995.
  6. Bayesian analysis of errors-in-variables regression models. Biometrics, pages 1085–1095, 1995.
  7. Inverse KKT: Learning cost functions of manipulation tasks from demonstrations. The International Journal of Robotics Research, 2017.
  8. A pivoting algorithm for linear programming with linear complementarity constraints. Optimization Methods and Software, 27(1):89–114, 2012.
  9. Handbook of Markov decision processes: methods and applications, volume 40. Springer Science & Business Media, 2012.
  10. Guided cost learning: Deep inverse optimal control via policy optimization. In International conference on machine learning, pages 49–58. PMLR, 2016.
  11. Christodoulos A. Floudas. Deterministic Global Optimization. Springer, 2000.
  12. Bayesian Data Analysis. Chapman and Hall/CRC, 2nd ed. edition, 2004.
  13. Bayesian Data Analysis. Chapman and Hall/CRC, 2013.
  14. Multiplier convergence in trust-region methods with application to convergence of decomposition methods for mpecs. Mathematical Programming, 112:335–369, 2008.
  15. Markov Chain Monte Carlo in Practice. Chapman and Hall/CRC, New York, December 1995.
  16. An analysis of the total least squares problem. SIAM journal on numerical analysis, 17(6):883–893, 1980.
  17. Zvi Griliches. Errors in variables and other unobservables. Econometrica: Journal of the Econometric Society, pages 971–998, 1974.
  18. Wilfred K. Hastings. Monte Carlo Sampling Methods Using Markov Chains and Their Applications. Biometrika, 57(1):97–109, 1970. Publisher: Oxford University Press, Biometrika Trust.
  19. Estimating parameters in optimal control problems. SIAM Journal on Scientific Computing, 34, 2012.
  20. A new regularization scheme for mathematical programs with complementarity constraints. SIAM Journal on Optimization, 20(1):78–103, 2009.
  21. A new regularization method for mathematical programs with complementarity constraints with strong convergence properties. SIAM Journal on Optimization, 23(2):770–798, 2013.
  22. Approximate bayesian inference in conditionally independent hierarchical models (parametric empirical bayes models). Journal of the American Statistical Association, pages 717–726, 1989.
  23. Model predictive control. Switzerland: Springer International Publishing, 38, 2016.
  24. Nonlinear programming. In Traces and emergence of nonlinear programming. Springer, 2014.
  25. A globally convergent filter method for MPECs. Preprint ANL/MCS-P1457-0907, Argonne National Laboratory, Mathematics and Computer Science Division, 2007.
  26. Objective learning from human demonstrations. Annual Reviews in Control, 2021.
  27. Maximum likelihood methods for inverse learning of optimal controllers. IFAC-PapersOnLine, 2020.
  28. Constrained inverse optimal control with application to a human manipulation task. IEEE Transactions on Control Systems Technology, 2019.
  29. Equation of State Calculations by Fast Computing Machines. The Journal of Chemical Physics, 21(6):1087–1092, December 1953.
  30. Yumiharu Nakano. Inverse stochastic optimal controls. Automatica, 149, 2023.
  31. An interior point method for mathematical programs with complementarity constraints (mpccs). SIAM Journal on Optimization, pages 720–750, 2005.
  32. Rajesh Rajamani. Vehicle dynamics and control. Springer Science & Business Media, 2011.
  33. Monte Carlo Statistical Methods. Springer Texts in Statistics. Springer, New York, NY, 2004.
  34. Stefan Scholtes. Convergence properties of a regularization scheme for mathematical programs with complementarity constraints. SIAM Journal on Optimization, pages 918–936, 2001.
  35. Torsten Söderström. A user perspective on errors-in-variables methods in system identification. Control Engineering Practice, 89:56–69, August 2019.
  36. Luke Tierney. Markov Chains for Exploring Posterior Distributions. The Annals of Statistics, 22(4):1701–1728, December 1994. Publisher: Institute of Mathematical Statistics.
  37. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical programming, 2006.
  38. Maximum entropy inverse reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, volume 8, 2008.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now