Finite-Horizon LQR for General Markov Jump Linear Systems: Deterministic Reformulation and Reduced-Order Solution

Abstract: This paper studies the Linear Quadratic Regulator (LQR) problem for continuous-time Markov Jump Linear Systems (MJLS) governed by general finite-state Markov chains that may include transient, absorbing, or non-communicating states. The problem, posed over a finite time horizon, is reformulated deterministically by expressing the cost functional in terms of a collection of second-moment matrices of the system state. A projection operator is introduced to restrict the analysis to the subspace of visited states, namely those with positive probability of being reached within the time horizon. The solution of the resulting deterministic problem is obtained from a reduced-order system of coupled matrix Riccati differential equations involving only the visited states, which define a quadratic value function satisfying a Hamilton-Jacobi-Bellman type equation. The structure of this equation is formally justified in the matrix setting via the Riesz-Frechet representation theorem, establishing a rigorous foundation for the deterministic reformulation and resolving an open aspect in previous literature. Several numerical examples, including cases with non-communicating states, validate the theoretical results and illustrate the practical relevance of the proposed generalization.