On the Partial Identifiability in Reward Learning: Choosing the Best Reward (2501.06376v1)

Abstract: In Reward Learning (ReL), we are given feedback on an unknown target reward, and the goal is to use this information to find it. When the feedback is not informative enough, the target reward is only partially identifiable, i.e., there exists a set of rewards (the feasible set) that are equally-compatible with the feedback. In this paper, we show that there exists a choice of reward, non-necessarily contained in the feasible set that, depending on the ReL application, improves the performance w.r.t. selecting the reward arbitrarily among the feasible ones. To this aim, we introduce a new quantitative framework to analyze ReL problems in a simple yet expressive way. We exemplify the framework in a reward transfer use case, for which we devise three provably-efficient ReL algorithms.