Automatic derivation of a compositional structure for a monolithic MDP

Determine an algorithmic method to compute, from a given monolithic Markov decision process, a string-diagram decomposition into open MDP components connected by the sequential composition and sum operators that faithfully represents the original model and is suitable for compositional model checking.

Background

The paper develops compositional value iteration for string diagrams of MDPs, where efficiency comes from reusing analyses of component open MDPs. This approach presumes that a suitable compositional structure of the system—captured as a string diagram—is available.

While many application domains naturally provide such hierarchical or compositional structures, the general problem of obtaining such a structure automatically from a monolithic MDP model is unresolved and would significantly broaden applicability of the proposed framework.