Evaluating surgical skills from kinematic data using convolutional neural networks (1806.02750v1)

Abstract: The need for automatic surgical skills assessment is increasing, especially because manual feedback from senior surgeons observing junior surgeons is prone to subjectivity and time consuming. Thus, automating surgical skills evaluation is a very important step towards improving surgical practice. In this paper, we designed a Convolutional Neural Network (CNN) to evaluate surgeon skills by extracting patterns in the surgeon motions performed in robotic surgery. The proposed method is validated on the JIGSAWS dataset and achieved very competitive results with 100% accuracy on the suturing and needle passing tasks. While we leveraged from the CNNs efficiency, we also managed to mitigate its black-box effect using class activation map. This feature allows our method to automatically highlight which parts of the surgical task influenced the skill prediction and can be used to explain the classification and to provide personalized feedback to the trainee.

• The paper presents a CNN that automatically evaluates surgical skills using multivariate kinematic data from robotic surgery.
• It reports 100% accuracy in suturing and needle passing and 92% accuracy for knot tying compared to traditional methods.
• The approach integrates Global Average Pooling and CAM to provide transparent, actionable feedback for surgical training.

Evaluating Surgical Skills from Kinematic Data using Convolutional Neural Networks

The paper "Evaluating surgical skills from kinematic data using convolutional neural networks" by Hassan Ismail Fawaz et al. presents an innovative approach to assessing surgical skills through the use of Convolutional Neural Networks (CNNs) applied to kinematic data derived from robotic surgery. The paper is grounded in the context of traditional surgical training methods, which rely largely on subjective observation and feedback. This research addresses the demand for an automatic and objective evaluation technique for surgical performances, free from biases related to manual assessments.

Methodology

The core of this paper lies in the design of a CNN specifically tailored for surgical skill evaluation using kinematic data. The authors employ the JIGSAWS dataset, which comprises kinematic data from surgical tasks performed on the da Vinci surgical system by surgeons with varying experience levels. With a focus on suturing, needle passing, and knot tying tasks, the CNN processes multivariate time series data to classify the skill levels of surgical performance into novice, intermediate, and expert categories.

The architecture integrates one-dimensional filters across kinematic variables while avoiding the dependency on predefined gesture boundaries. This method harnesses domain knowledge to group input channels into clusters and sub-clusters based on correlations among kinematic variables. The CNN has hierarchically structured layers, with the last layer utilizing Global Average Pooling (GAP) to facilitate the Class Activation Map (CAM) process, enabling transparent skill assessment and personalized feedback.

Results

The paper demonstrates the efficacy of the proposed CNN model through comprehensive testing. Notably, the model achieves 100% accuracy in classifying surgical skills for suturing and needle passing tasks and a commendable accuracy rate of around 92% for the complex knot tying activities. These results are compared against other methods such as Sparse Hidden Markov Models (S-HMM), Approximate Entropy (ApEn), and Sax-Vsm approaches, highlighting the superior performance and additional interpretability offered by this CNN method. Unlike the other models, this approach provides detailed feedback that can aid trainees in honing their skills effectively based on visualized heatmaps derived from CAM.

Implications and Future Work

The implications of this research are multifaceted, offering both practical enhancements to surgical training and advancing theoretical understanding of skill assessment methodologies. By mitigating the "black-box" nature typically associated with deep learning models, the system presents a path forward in developing AI that holds potential for widespread implementation in clinical training environments.

Looking ahead, the authors propose further exploration into merging kinematic data with video inputs to create a holistic framework for surgical skill classification. Such developments may increase the granularity of feedback and improve the accuracy of evaluations, driving the evolution of AI applications in the domain of Surgical Data Science.

In summary, the paper presents a significant contribution to the field by introducing a robust, accurate, and interpretable method for evaluating surgical skills using CNNs, revealing a promising avenue for improving surgical education with AI-powered automation.