The value of Markov Chain Games with incomplete information on both sides

Abstract: We consider zero-sum repeated games with incomplete information on both sides, where the states privately observed by each player follow independent Markov chains. It generalizes the model, introduced by Aumann and Maschler in the sixties and solved by Mertens and Zamir in the seventies, where the private states of the players were fixed. It also includes the model introduced in Renault \cite{R2006}, of Markov chain repeated games with lack of information on one side, where only one player privately observes the sequence of states. We prove here that the limit value exists, and we obtain a characterization via the Mertens-Zamir system, where the "non revealing value function" plugged in the system is now defined as the limit value of an auxiliary "non revealing" dynamic game. This non revealing game is defined by restricting the players not to reveal any information on the {\it limit behavior} of their own Markov chain, as in Renault 2006. There are two key technical difficulties in the proof: 1) proving regularity, in the sense of equicontinuity, of the $T$-stage non revealing value functions, and 2) constructing strategies by blocks in order to link the values of the non revealing games with the original values.