Uniform mixing and coverage conditions for the primal–dual policy sequence

Establish sufficient conditions under which the policy sequence generated by the Primal–Dual Politex algorithm for constrained partially observable Markov decision processes satisfies the uniform mixing and coverage properties required for the linear temporal-difference estimation guarantees.

Background

The paper implements its estimation oracle with projected linear TD learning over finite history windows. The resulting finite-sample error bound is conditional on constants that hold uniformly over every policy generated by the outer Primal–Dual Politex algorithm and over all iteration budgets. These constants encode persistent exploration, mixing, and coverage properties of the sampling process.

The exponential-weights policy update does not itself guarantee persistent exploration. Consequently, the paper leaves unresolved the task of identifying conditions on the constrained partially observable Markov decision process, feature representation, or algorithmic policy sequence that would ensure the required uniform mixing and coverage properties and thereby make the sample-complexity guarantee unconditional.