Reinforcement Learning for Inverse Non-Cooperative Linear-Quadratic Output-feedback Differential Games

Abstract: In this paper, we address the inverse problem for linear-quadratic differential non-cooperative games with output-feedback. Given players' stabilizing feedback laws, the goal is to find cost function parameters that lead to a game for which the observed game dynamics are at a Nash equilibrium. Using the given feedback laws, we introduce a model-based algorithm that generates cost function parameters solving the above inverse problem. We introduce a correction procedure that at each iteration of the algorithm guarantees the existence of the feedback laws, which addresses a key challenge of output-feedback control designs. As an intermediate stage of the algorithm, we have developed a procedure for the initial stabilization of the multiple-input system with output-feedback information structure. We prove convergence and stability of the algorithm, and show the way to generate new games with necessary properties without requiring to run the complete algorithm repeatedly. Then the algorithm is extended to a model-free version that uses data samples generated by unknown dynamics and has the same converging and stabilizing properties as the model-based version. Finally, we show how the inverse problem can be solved in a distributed manner and provide possible extensions. Simulation results validate the effectiveness of the proposed algorithms.