Methodical evaluation of ethical behavior in sequential decision-making systems

Develop rigorous, methodical empirical evaluation frameworks that assess the ethical behavior of deployed sequential decision-making systems modeled as Markov decision processes (MDPs) and their variants, including systematically identifying which stakeholders are harmed and quantifying harms in real-world operation.

Background

The paper contrasts evaluation practices for predictive models with those for sequential decision-making systems, noting that many harms and shortcomings in sequential settings are only discoverable during real-world deployment. Existing evaluation methods tend to focus on model-based metrics or simulation, which do not reveal the actual impacts on stakeholders. The authors emphasize the lack of rigorous empirical studies of harms from deployed sequential decision-making systems and the need for structured evaluation approaches.

This gap is particularly acute because sequential decision-making systems act and enact change over time, making it challenging to anticipate and measure ethical outcomes without deployment. Establishing methodical, real-world evaluation procedures is therefore presented as a critical, yet unresolved, research problem.