Communicating Robot Conventions through Shared Autonomy (2202.11140v2)

Abstract: When humans control robot arms these robots often need to infer the human's desired task. Prior research on assistive teleoperation and shared autonomy explores how robots can determine the desired task based on the human's joystick inputs. In order to perform this inference the robot relies on an internal mapping between joystick inputs and discrete tasks: e.g., pressing the joystick left indicates that the human wants a plate, while pressing the joystick right indicates a cup. This approach works well after the human understands how the robot interprets their inputs -- but inexperienced users still have to learn these mappings through trial and error! Here we recognize that the robot's mapping between tasks and inputs is a convention. There are multiple, equally efficient conventions that the robot could use: rather than passively waiting for the human, we introduce a shared autonomy approach where the robot actively reveals its chosen convention. Across repeated interactions the robot intervenes and exaggerates the arm's motion to demonstrate more efficient inputs while also assisting for the current task. We compare this approach to a state-of-the-art baseline -- where users must identify the convention by themselves -- as well as written instructions. Our user study results indicate that modifying the robot's behavior to reveal its convention outperforms the baselines and reduces the amount of time that humans spend controlling the robot. See videos of our user study here: https://youtu.be/jROTVOp469I

Communicating Robot Conventions through Shared Autonomy

The paper by Ananth Jonnavittula and Dylan P. Losey explores an innovative approach to enhancing human-robot interaction by communicating robot conventions to human users through shared autonomy. This research addresses a critical challenge in assistive teleoperation where robots need to accurately infer human intent, which relies on an internal mapping between user inputs (via joysticks) and specific tasks. The authors propose a methodology where robots actively reveal their chosen conventions to guide humans towards more efficient interactions, contrasting with traditional systems where humans must learn these mappings independently.

Contributions and Approach

The paper makes three primary contributions:

1. Formalizing Conventions in Shared Autonomy: The authors define conventions as mappings between high-level tasks and low-level inputs. These mappings are essential for robots to interpret human inputs, but there exists an equivalence of multiple, equally efficient conventions. The paper formalizes the role of these conventions in task inference in shared autonomy settings and introduces the concept of exaggerated actions — intentional deviations from direct paths that are more informative for robot intent inference.
2. Communicating Conventions over Repeated Interaction: The paper proposes a method where robots leverage shared autonomy to guide users through an experiential learning process. By intervening and exaggerating robot actions, these systems provide implicit feedback that helps users learn optimal input strategies over repeated interactions. The authors prove that this demonstration-based teaching method is more efficient than requiring users to deduce conventions independently.
3. Comparison to Written Instructions: Through user paper evaluations, the paper demonstrates that the proposed approach outperforms others, including relying on written instructions, especially when conventions are complex or unintuitive. The paper reveals that direct demonstrations using robot motions are more effective than documentation in teaching users how to communicate tasks accurately and concisely.

Methodology and Results

The methodology hinges on a constrained optimization approach that balances revealing actions that communicate conventions and actions that assist users in task completion. This optimization considers the robot's confidence in inferred user intents, helping it adjust its actions to reveal more about the task it believes the user intends to convey. The proposed method is tested through a user paper involving object selection tasks using a robot arm under various conditions, including straightforward and complex task mappings.

Results from this paper indicate that the shared autonomy approach significantly improves users' ability to convey their intended tasks with fewer inputs, suggesting enhanced efficiency and user experience. Notably, participants interacting with robots capable of demonstrating conventions in real-time adapted more rapidly compared to those relying on conventional methods.

Implications and Future Directions

The research implications extend to both practical applications and theoretical insights within AI and robotics. Practically, this shared autonomy approach can considerably improve user experience and efficacy in assistive robotics, ultimately facilitating broader adoption in settings ranging from healthcare to personal robotics. Theoretically, the paper contributes to our understanding of dynamic human-robot interaction, suggesting avenues for future research in adaptive systems that learn and impart user-specific conventions over time.

Potential future developments could explore more sophisticated models of user adaptation and learning rates, investigate methods to personalize conventions based on individual user preferences, and examine long-term effects of this teaching method on task performance in various real-world scenarios. Additionally, integrating multimodal feedback beyond visual cues could provide a richer learning environment for users.

Overall, this paper presents a compelling case for active convention teaching in robotic teleoperation, inviting further research into human-centered approaches that advance both the utility and intuitiveness of robot-assisted systems.