Feasibility of interventions across MDP variants

Ascertain the feasibility of different mitigation interventions across extensions of Markov decision processes—including partially observable Markov decision processes (POMDPs), decentralized MDPs (DecMDPs), adversarial reinforcement learning settings, and hierarchical decision processes—given their differing assumptions and complexity.

Background

The paper describes numerous extensions of MDPs used to model diverse sequential decision problems, each with distinct informational and structural properties. Because these models vary significantly, it is uncertain which interventions for ethical behavior—such as state-space augmentation, reward modification, or constraint-based formulations—are feasible and effective across model classes.

The authors explicitly flag this as an open question, underscoring the need for systematic analysis of intervention portability and limitations across different sequential decision-making paradigms.