Design and Evaluation of an Uncertainty-Aware Shared-Autonomy System with Hierarchical Conservative Skill Inference

Abstract: Shared-autonomy imitation learning lets a human correct a robot in real time, mitigating covariate-shift errors. Yet existing approaches ignore two critical factors: (i) the operator's cognitive load and (ii) the risk created by delayed or erroneous interventions. We present an uncertainty-aware shared-autonomy system in which the robot modulates its behaviour according to a learned estimate of latent-space skill uncertainty. A hierarchical policy first infers a conservative skill embedding and then decodes it into low-level actions, enabling rapid task execution while automatically slowing down when uncertainty is high. We detail a full, open-source VR-teleoperation pipeline that is compatible with multi-configuration manipulators such as UR-series arms. Experiments on pouring and pick-and-place tasks demonstrate 70-90% success in dynamic scenes with moving targets, and a qualitative study shows a marked reduction in collision events compared with a non-conservative baseline. Although a dedicated ablation that isolates uncertainty is impractical on hardware for safety and cost reasons, the reported gains in stability and operator workload already validate the design and motivate future large-scale studies.

• The paper presents a novel system integrating uncertainty inference with conservative skill strategies to enhance robot performance and safety.
• It introduces a hierarchical skill network that separates high- and low-level policies for effective skill embedding and abstraction in dynamic scenarios.
• Experimental results show success rates up to 90% in static and 80% in dynamic tasks, highlighting the system's robustness and scalability.

Design and Evaluation of an Uncertainty-Aware Shared-Autonomy System

The paper "Design and Evaluation of an Uncertainty-Aware Shared-Autonomy System with Hierarchical Conservative Skill Inference" focuses on developing an advanced shared autonomy system designed to enhance the interaction of robots with dynamic environments while incorporating human guidance effectively. The proposed system emphasizes uncertainty inference and conservative skill strategies to mitigate potential errors during the learning process.

Introduction

Shared autonomy systems have been instrumental in enabling robots to learn from human demonstrations and corrections. These systems share the workspace with humans, allowing robots to correct actions despite unseen states and reduce compounding errors through expert interventions. However, they require constant human supervision which can lead to operator fatigue, judgment errors, and untimely interventions, thus posing safety risks. The paper introduces an uncertainty-aware system for inferred conservative task execution that responds to these challenges.

Figure 1: Control loop for a Hierarchical Conservative Skill Network in Uncertainty-Aware Shared Autonomy Process.

Hierarchical Skill Network Design

The hierarchical skill network (HSN) introduced in this study facilitates learning abstract skill levels rather than direct end-effector trajectories. Inspired by SPiRL, the system comprises high- and low-level policies to create skill embeddings and translate them into actual robot behavior. This structure not only ensures systematic multi-skilled learning but also improves scalability in dynamic environments.

Figure 2: Overall architecture of the hierarchical skill network (HSN).

To infer uncertainties, Monte-Carlo dropout is applied to the skill prior network, allowing for a conservative planning approach based on the calculated skill uncertainty. The integration of conservative inference into policymaking reduces the likelihood of task execution failures, enhancing stability during both training and operational phases.

Implementation Details

The shared autonomy process is characterized by a comprehensive system configuration, including components essential for VR teleoperation and real-time system interactions.

Figure 3: The comprehensive structure of our SAP system and the configuration of the VR controller.

The VR teleoperation allows for real-time control adjustments without the necessity for head-mounted displays, enabling operators to adjust robot configurations for multi-task learning. This system employs constrained teleoperation techniques to mitigate collision risks by implementing virtual motion constraints.

Experiments and Results

Experiments involving UR3 robots equipped with VR devices and cameras demonstrate the system's capacity to learn several manipulation skills, showcasing a robust adaptability to dynamic environmental transformations.

With success rates reaching 90% for pouring tasks in static environments and 80% in dynamic scenarios, the hierarchical skill network demonstrated efficient skill embedding and inference capabilities. However, pick-and-place tasks encountered challenges due to occlusions, which indicates potential improvements with augmented observation tools.

Figure 4: Evaluation of adaptable manipulation skills in dynamic environments where humans manipulate target objects to simulate variations.

Conclusions

The proposed uncertainty-aware shared autonomy system, with its conservative skill inference methodology, significantly enhances the stability and adaptability of robots in dynamic environments. By leveraging hierarchical skill networks and incorporating human feedback, it reconciles the practicality of autonomous systems with the necessity of human oversight.

Future advancements could explore integrating patch-based video understanding methods, such as Vision Transformers, to refine uncertainty estimation and improve skill accuracy in variable conditions.

Implications and Future Work

This study highlights critical advancements in shared autonomy systems, paving the way for more resilient AI-Powered robotics capable of seamless human collaboration in unpredictable environments. Future research should focus on refining uncertainty estimation techniques and exploring multi-sensory integration to further enhance environmental adaptability and task execution precision.