Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Corrected: On Confident Policy Evaluation for Factored Markov Decision Processes with Node Dropouts (2302.02297v1)

Published 5 Feb 2023 in eess.SY and cs.SY

Abstract: In this work we investigate an importance sampling approach for evaluating policies for a structurally time-varying factored Markov decision process (MDP), i.e. the policy's value is estimated with a high-probability confidence interval. In particular, we begin with a multi-agent MDP controlled by a known policy but with unknown transition dynamics. One agent is then removed from the system - i.e. the system experiences node dropout - forming a new MDP of the remaining agents, with a new state space, action space, and new transition dynamics. We assume that the effect of removing an agent corresponds to the marginalization of its factor in the transition dynamics. The reward function may likewise be marginalized, or it may be entirely redefined for the new system. Robust policy importance sampling is then used to evaluate candidate policies for the new system, and estimated values are presented with probabilistic confidence bounds. This computation is completed with no observations of the new system, meaning that a safe policy may be found before dropout occurs. The utility of this approach is demonstrated in simulation and compared to Monte Carlo simulation of the new system.

Summary

We haven't generated a summary for this paper yet.