The recent advances in deep learning and computer vision have led to a growing number of studies focusing on the automatic analysis of surgical video data. Since the use of video is an integral part of minimally invasive surgery (MIS), most of these studies have focused on laparoscopic and robotic surgery. In contrast, video capture is not well established in open surgery. Thus, open surgery has not benefited from the many advantages computer vision and deep learning methods have to offer for skill training and automatic assistance. Furthermore, while using simulators is inherent to training MIS, assessing open surgery skills, which is no less important, is lagging. This study focuses on both the development of novel video analysis algorithms as well as using these algorithms for assessment of surgical skills.
To request Dataset:
Medical simulators provide a controlled environment for training and assessing clinical skills. However, as an assessment platform, it requires the presence of an experienced examiner to provide performance feedback, commonly performed using a task specific checklist. This makes the assessment process inefficient and expensive. Furthermore, this evaluation method does not provide medical practitioners the opportunity for independent training.
To this end, we have developed an autonomous and a fully automatic speech-based checklist system, capable of objectively identifying and validating anesthesia residents’ actions in a simulation environment. In the future, this approach may be implemented in the operation room and emergency room. This could facilitate the development of automatic assistive technologies for these domains.
The Fundamental Laparoscopic Skills (FLS) teaches the knowledge, judgment, and technical skills that are expected in laparoscopic surgery. It consists of a cognitive knowledge component, as well as a skills-based portion pertinent to laparoscopic surgery.
As part of our research, we evaluated two essential procedures in the exam: sewing and peg transfer. The goal of this work is use computer vision to develop automatic feedback based on video data alone. The automatic feedback will allow residents to practice on their own without the need for a supervising human expert.
This work is done in collaboration with Dr. Gideon Sroka from Bnai-Zion Medical Center.
