Evaluating the Impact of Personalized Value Alignment in Human-Robot Interaction: Insights into Trust and Team Performance Outcomes (2311.16051v1)

Abstract: This paper examines the effect of real-time, personalized alignment of a robot's reward function to the human's values on trust and team performance. We present and compare three distinct robot interaction strategies: a non-learner strategy where the robot presumes the human's reward function mirrors its own, a non-adaptive-learner strategy in which the robot learns the human's reward function for trust estimation and human behavior modeling, but still optimizes its own reward function, and an adaptive-learner strategy in which the robot learns the human's reward function and adopts it as its own. Two human-subject experiments with a total number of 54 participants were conducted. In both experiments, the human-robot team searches for potential threats in a town. The team sequentially goes through search sites to look for threats. We model the interaction between the human and the robot as a trust-aware Markov Decision Process (trust-aware MDP) and use Bayesian Inverse Reinforcement Learning (IRL) to estimate the reward weights of the human as they interact with the robot. In Experiment 1, we start our learning algorithm with an informed prior of the human's values/goals. In Experiment 2, we start the learning algorithm with an uninformed prior. Results indicate that when starting with a good informed prior, personalized value alignment does not seem to benefit trust or team performance. On the other hand, when an informed prior is unavailable, alignment to the human's values leads to high trust and higher perceived performance while maintaining the same objective team performance.

• The paper demonstrates that adaptive learner robots significantly enhance trust and reliance when they adjust to human values through IRL.
• It employs controlled experiments comparing informed and uninformed priors to assess the effects of value alignment on team performance.
• Findings suggest that robots adopting human values can optimize trust dynamics without compromising objective task outcomes.

Understanding Human-Robot Value Alignment

Introduction to Value Alignment in HRI

Robots are entering various facets of human society, assuming roles that require them to work in tandem with humans. As such, the ability for robots to align their values with human teammates becomes crucial, particularly to foster trust and enhance team performance. This paper sheds light on the dynamic relationship between human-robot value alignment and its impact on trust and performance within a team setting.

The Experiment and Its Design

The paper explores the interaction between humans and robots using three distinct robot strategies:

• The Non-Learner strategy, where the robot assumes that the human shares its values without learning or adapting.
• The Non-Adaptive Learner strategy, where the robot learns the human's values but still prioritizes its own during decision-making.
• The Adaptive Learner strategy, where the robot not only learns the human's values but also adopts them as its own.

Two separate experiments were conducted involving participants tasked with finding threats in a virtual environment under a risk of exposure. Different from prior studies, which usually presuppose value alignment as beneficial, this paper presents an empirical evaluation of its effects. Through the lens of a trust-aware Markov Decision Process (MDP) and Bayesian Inverse Reinforcement Learning (IRL), the adaptive strategies were compared against a backdrop of informed and uninformed priors on human values.

Results and Insights

• Experiment 1 (Informed Prior): When the IRL learning algorithm was initiated with a priori knowledge about human reward weights, no notable benefits were seen across the board for the learning and adaptive strategies. The only exception was a higher stated reliance intention for the non-learner approach compared to the adaptive learner. Surprisingly, a more predictable (non-adaptive) robot inclined users toward increased reliance.
• Experiment 2 (Uninformed Prior): Without an informed prior, the advantages of value alignment became evident. The adaptive-learner strategy exhibited a significant increase in trust, agreement with robot recommendations, reliance intentions, and perceived team performance, without sacrificing objective performance.

Implications and Concluding Thoughts

The findings are pivotal for real-world applications where robots interact with humans, especially in scenarios where a robot's initial understanding of human values is not accessible. Ensuring that robots can adapt their values to those of their human counterparts can lead to improved trust dynamics and team outcomes. This research contributes to understanding how human-robot partnerships may function optimally, emphasizing the need for robots that can understand and align with human values.

This paper is also mindful of its scope and limitations, encouraging further research to generalize findings across diverse demographics and complex decision-making situations. The necessity for adaptable and trust-conscious robots in a collaborative future is clear; how we mold these artificial entities to reflect human moral and ethical systems will be the subject of continued and necessary debate and investigation.