2000 character limit reached
Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shaping
Published 15 Dec 2023 in cs.LG, cs.AI, and stat.ML | (2312.09983v2)
Abstract: Inverse reinforcement learning (IRL) is computationally challenging, with common approaches requiring the solution of multiple reinforcement learning (RL) sub-problems. This work motivates the use of potential-based reward shaping to reduce the computational burden of each RL sub-problem. This work serves as a proof-of-concept and we hope will inspire future developments towards computationally efficient IRL.
- Apprenticeship learning via inverse reinforcement learning. In Proceedings of the Twenty-First International Conference on Machine Learning, pp. 1–8, 2004.
- Identifiability in inverse reinforcement learning. In Proceedings of the 35th Conference on Neural Information Processing Systems, pp. 1–11, 2021.
- Heuristic-guided reinforcement learning. arXiv, pp. 2106.02757, 2021.
- Guaranteeing control requirements via reward shaping in reinforcement learning. arXiv, pp. 2311.10026, 2023.
- Principled reward shaping for reinforcement learning via Lyapunov stability theory. Neurocomputing, 393:83–90, 2020.
- Unpacking reward shaping: Understanding the benefits of reward engineering on sample complexity. arXiv, pp. 2210.09579, 2022.
- Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping. arXiv, pp. 2011.02669, 2020.
- Sham M. Kakade. On the Sample Complexity of Reinforcement Learning. PhD thesis, University College London, 2003.
- Adam: A method for stochastic optimization. arXiv, pp. 1412.6980, 2014.
- Bridging RL Theory and Practice with the Effective Horizon. arXiv, pp. 2304.09853, 2023.
- James MacGlashan and Michael L. Littman. Between imitation and intention learning. In Qiang Yang and Michael Wooldridge (eds.), Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, pp. 3692–3698, 2015.
- Human-level control through deep reinforcement learning. Nature, 518:529–533, 2015.
- Andrew Y. Ng and Stuart J. Russell. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pp. 663–670, 2000.
- Policy invariance under reward transformations: Theory and application to reward shaping. In Proceedings of the Sixteenth International Conference on Machine Learning, pp. 278–287, 1999.
- Bayesian inverse reinforcment learning. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp. 2586–2591, 2007.
- Stuart Russell. Learning Agents for Uncertain Environments (Extended Abstract). In Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 101–103. Association for Computing Machinery, 1998. ISBN 1581130570.
- Proximal policy optimization algorithms. arXiv, pp. 1707.06347, 2017.
- Reinforcement Learning, second edition: An Introduction. Adaptive Computation and Machine Learning series. MIT Press, 2018. ISBN 9780262352703.
- Maximum entropy deep inverse reinforcement learning. arXiv, pp. 1507.04888, 2016.
- Receding horizon inverse reinforcement learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Proceedings of the Thirty-Sixth Conference on Neural Information Processing Systems, 2022.
- Maximum entropy inverse reinforcement learning. In Anthony Cohn (ed.), Proceedings of the 23rd National Conference on Artificial Intelligence, pp. 1433–1438, 2008.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.