SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning (2505.10251v2)

Abstract: Research on autonomous surgery has largely focused on simple task automation in controlled environments. However, real-world surgical applications demand dexterous manipulation over extended durations and robust generalization to the inherent variability of human tissue. These challenges remain difficult to address using existing logic-based or conventional end-to-end learning strategies. To address this gap, we propose a hierarchical framework for performing dexterous, long-horizon surgical steps. Our approach utilizes a high-level policy for task planning and a low-level policy for generating low-level trajectories. The high-level planner plans in language space, generating task or corrective instructions to guide the robot through the long-horizon steps and correct for the low-level policy's errors. We validate our framework through ex vivo experiments on cholecystectomy, a commonly-practiced minimally invasive procedure, and conduct ablation studies to evaluate key components of the system. Our method achieves a 100% success rate across n=8 different ex vivo gallbladders, operating fully autonomously without human intervention. The hierarchical approach improves the policy's ability to recover from suboptimal states that are inevitable in the highly dynamic environment of realistic surgical applications. This work demonstrates step-level autonomy in a surgical procedure, marking a milestone toward clinical deployment of autonomous surgical systems.

• The paper introduces SRT-H, a hierarchical framework utilizing language-conditioned imitation learning for autonomous robotic surgery.
• The SRT-H framework achieved a 100% success rate across eight ex-vivo cholecystectomy procedures without any human intervention.
• This hierarchical approach enhances robustness and resilience in complex surgical environments, paving the way for more reliable robotic systems in clinical settings.

An Examination of the SRT-H Framework for Autonomous Surgery

The paper "SRT-H: Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learning," authored by Ji Woong (Brian) Kim et al., presents a novel approach to achieving autonomous robotic surgery in complex environments. The hierarchical framework, SRT-H, is designed to address the challenges of dexterous manipulation and long-horizon task management in surgical procedures, notably cholecystectomy.

The significance of this research lies in its ability to provide a hierarchical structure to robotic surgical operations. Autonomous surgery represents a substantial leap in improving surgical outcomes and accessibility. However, the complexities of surgical scenes—characterized by dynamic environments and variabilities in human tissues—pose significant challenges for automation. This framework aims to overcome these obstacles by employing both high-level (HL) and low-level (LL) policies in the surgical robot.

Task Execution and Validation

The SRT-H framework uses natural language as a modality for task planning, mirroring how surgeons instruct trainees verbally. This approach enables more intuitive interaction with the surgical robot, allowing it to perform surgical tasks autonomously while adhering to human-like instruction methods. The HL policy generates coarse guidance via language instructions, whereas the LL policy focuses on precise task-space controls necessary for surgical manipulation. Notably, the SRT-H framework was validated extensively through ex-vivo cholecystectomy experiments, achieving a 100% success rate across eight different gallbladders without any human intervention.

Numerical Results and Demonstration

Key numerical results underscore the performance advantages of SRT-H in autonomously executing surgical tasks. The framework showed robust performance across diverse anatomical variabilities in gallbladder tissues, with an average task completion duration of 317 seconds and minimal corrective intervention. Ablative studies further demonstrated that the hierarchical approach contributed significantly to the system’s resilience and capability to recover from suboptimal states encountered in intricate surgical environments.

Implications for Surgical Robotics

The implications of this research extend beyond the immediate application in cholecystectomy. The hierarchical design and integration of language-guided policies pave the way for greater procedural autonomy in surgical robotics. By advancing generalizable autonomy, SRT-H could facilitate more reliable deployment of robotic systems in clinical settings. This approach can potentially be expanded to other long-horizon procedures, increasing the adaptability and coverage of robotic assistance in surgical operations.

Future Directions

The paper’s insights into integrating natural language processing with robotic control systems suggest promising directions for future developments in AI-driven surgery. There remains considerable scope for advancing sensor technologies and refining the decision-making process for intricate surgical maneuvers. A pertinent area for exploration is the translation of this ex-vivo validated framework to in-vivo clinical trials, which would require addressing additional challenges such as surgical field occlusion and tissue movement. Furthermore, the evolving landscape of AI tools, including advancements in LLMs and visual-LLMs (VLMs), indicates potential enhancements in interfacing and autonomy capabilities.

Conclusion

In conclusion, the authors provide compelling numeric evidence and insightful theoretical perspectives on the advancement of autonomous surgical systems through the SRT-H framework. By bridging gaps in existing robotic capabilities and expanding the autonomy horizon, this research contributes significantly to the field of surgical robotics. It heralds a critical milestone in the path toward achieving sophisticated and reliable autonomous surgical procedures.