Solving Poisson's Equation: Existence, Uniqueness, Martingale Structure, and CLT

Abstract: The solution of Poisson's equation plays a key role in constructing the martingale through which sums of Markov correlated random variables can be analyzed. In this paper, we study two different representations for the solution in countable state space, one based on regenerative structure and the other based on an infinite sum of expectations. We also consider integrability and related uniqueness issues associated with solutions to Poisson's equation, and provide verifiable Lyapunov conditions to support our theory. Our key results include a central limit theorem and law of the iterated logarithm for Markov dependent sums, under Lyapunov conditions weaker than have previously appeared in the literature.