A Numerical Truncation Approximation with A Posteriori Error Bounds for the Solution of Poisson's Equation

Abstract: The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for equilibrium Markovian rewards. In this paper, we consider the problem of numerically computing the solution to Poisson's equation when the state space is infinite or very large. In such settings, the state space must be truncated in order to make the problem computationally tractable. In this paper, we provide the first truncation approximation solution to Poisson's equation that comes with provable and computable a posteriori error bounds. Our theory applies to both discrete-time chains and continuous-time jump processes. Through numerical experiments, we show our method can provide highly accurate solutions and tight bounds.