Lifting the polytopic restriction via near‑optimal policies in arbitrary RMDPs

Investigate whether the restriction to polytopic uncertainty sets in the implicit anytime algorithm for robust Markov decision processes without the Constant-Support Assumption can be lifted by characterizing near‑optimal policies in arbitrary uncertainty sets so as to obtain the stable strategy recommender required for the algorithm’s correctness.

Background

To ensure convergence of the SEC-based deflation/inflation method, the algorithm requires a stable strategy recommender, which in turn depends on the existence of memoryless deterministic optimal policies. This is guaranteed for polytopic uncertainty sets but may fail for arbitrary uncertainty sets without Constant-Support.

The authors therefore currently restrict to polytopic RMDPs in this part of their theory and propose lifting this restriction by analyzing the structure of near-optimal policies in arbitrary RMDPs.