Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot (2002.06302v1)

Abstract: Recent advances in depth-sensing have significantly increased accuracy, resolution, and frame rate, as shown in the 1920x1200 resolution and 13 frames per second Zivid RGBD camera. In this study, we explore the potential of depth sensing for efficient and reliable automation of surgical subtasks. We consider a monochrome (all red) version of the peg transfer task from the Fundamentals of Laparoscopic Surgery training suite implemented with the da Vinci Research Kit (dVRK). We use calibration techniques that allow the imprecise, cable-driven da Vinci to reduce error from 4-5 mm to 1-2 mm in the task space. We report experimental results for a handover-free version of the peg transfer task, performing 20 and 5 physical episodes with single- and bilateral-arm setups, respectively. Results over 236 and 49 total block transfer attempts for the single- and bilateral-arm peg transfer cases suggest that reliability can be attained with 86.9 % and 78.0 % for each individual block, with respective block transfer speeds of 10.02 and 5.72 seconds. Supplementary material is available at https://sites.google.com/view/peg-transfer.

• The paper demonstrates that calibrating depth-sensing in a da Vinci robot reduces positioning errors from approximately 4-5mm to 1-2mm, improving precision in surgical tasks.
• The study employs a Zivid RGBD camera and bilateral interpolation for real-time detection and manipulation, achieving success rates of 86.9% in single-arm and 78.0% in bilateral-arm setups.
• The findings imply that integrating advanced depth-sensing with automated calibration and motion planning could significantly advance robotic surgical systems and error recovery techniques.

Automated Surgical Manipulation Using Depth-Sensing with the da Vinci Robot

In recent advancements, automated surgical procedures are increasingly garnering interest due to the potential to enhance surgical precision, reduce human error, and alleviate surgeon fatigue. This paper presents a paper where depth-sensing techniques are employed within an automated framework using the da Vinci Research Kit (dVRK) to facilitate the robotic execution of the peg transfer task—a standard assessment in the Fundamentals of Laparoscopic Surgery (FLS) training suite. The exploration of utilizing a Zivid RGBD camera, capable of a 1920x1200-resolution and a frame rate of 13 frames per second, offers promising insights into the integration of depth-sensing for surgical robotics.

Methodology

The research focuses on a monochrome variation of the classic peg transfer task. The task involves the manipulation of six triangular blocks across a pegboard, simulating conditions that mimic real surgical environments by painting all objects red, thereby testing the depth-sensing capabilities absent of color differentiation. A significant methodological contribution is the calibration approach, which compensates for the cable-driven nature of da Vinci robots that often introduces motion inaccuracies. This calibration reduces error from approximately 4-5mm to 1-2mm within the task workspace, a crucial adjustment for precise surgical manipulation.

The authors tested both single-arm and bilateral-arm setups. The tasks were divided into discrete motion sequences for grasping and placing blocks, with a robust depth-sensing algorithm guiding the robot's decisions. Notably, the algorithm employed bilateral interpolation within depth maps for detecting blocks and pegs, an approach requiring efficient computation to maintain the workflow within real-time constraints.

Experimental Results

Experiments conducted reported a success rate of 86.9% for single-arm operations and 78.0% for bilateral-arm cases. The average time for block transfer varied, recorded at 10.02 seconds for single-arm and 5.72 seconds for bilateral-arm setups, demonstrating a speed advantage when using two arms. These outcomes included diverse failure modes like pick failures, place-stuck, and place-fall failures, crucial metrics for understanding operational robustness. Interestingly, when these failures were corrected by subsequent actions, the success rates improved slightly to 89.0% and 83.1% for single and bilateral setups, respectively.

Implications and Future Directions

These findings represent a significant step in the journey toward surgical task automation. While the performance of the automated system lags behind that of expert human surgeons, particularly in dynamic error correction during task execution, the application of depth-sensing to robotic surgical systems introduces promising avenues for future research. Subsequent efforts would benefit from addressing the prevalent placing failures, possibly through integrating advanced motion-planning strategies, enhanced visual servoing, and tactile feedback mechanisms that exploit both depth-sensor data and joint motor currents for augmented error recovery.

Furthermore, utilizing surgical simulation environments to train reinforcement learning models for closed-loop control systems may drive advances in autonomy and efficiency, potentially narrowing the performance gap with expert human operators. The exploration into more sophisticated calibration methodologies and trajectory optimization techniques remains a fertile ground for ensuring precise robotic actuation in complex surgical tasks.

This work highlights the strategic value of incorporating evolving depth-sensing technologies into robotic surgical systems and lays down a framework for continued enhancement, addressing both theoretical challenges and practical implementations in medical robotics.