Limit Value in Zero-Sum Stochastic Games with Vanishing Stage Duration and Public Signals (2403.07467v2)

Abstract: We consider the behaviour of $\lambda$-discounted zero-sum games as the discount factor $\lambda$ approaches $0$ (that is, the players are more and more patient), in the context of games with stage duration. In stochastic games with stage duration $h$, players act at times $0, h, 2h,$ and so on. The payoff and leaving probabilities are proportional to $h$. When $h$ tends to $0$, such discrete-time games approximate games played in continuous time. The asymptotic behavior of the values (when both $\lambda$ and $h$ tend to $0$) was already studied in the case of stochastic games with perfect observation of the state and in the state-blind case. We consider the same question for the case of stochastic games with imperfect observation of the state. More precisely, we consider a particular case of such games, stochastic games with public signals, in which players are given at each stage a public signal that depends only on the current state. Our main result states that there exists a stochastic game with public signals, with no limit value (as the discount factor $\lambda$ goes to $0$) if stage duration is $1$, but with a limit value when stage duration $h$ and discount factor $\lambda$ both tend to $0$. Informally speaking, it means that the limit value in discrete time does not exist, but the limit value in continuous time (i.e. when $h$ approaches $0$) exists. Such a situation is impossible in the case of stochastic games with perfect observation of the state.