Abstract
Background
Visualization during minimally invasive bypass surgery on the beating heart can be enhanced by using a robotic‐guided endoscope and visual servoing from the endoscopic images. In order to achieve these objectives, this work has focused on developing and testing algorithms for accurate, robust and real‐time motion tracking of features on the beating heart, using marker‐less approaches and an uncalibrated endoscope.
Methods
Lucas–Kanade pyramidal optical flow‐based algorithms and speeded‐up robust features (SURF)‐based methods have been extensively evaluated, using a range of developed metrics, in order to quantify accuracy, robustness and drift under a variety of circumstances. Three sets of experiments are reported: the first set compared the two tracking methods, using a beating‐heart phantom and a static endoscope; the second set evaluated the methods when images were taken using a moving robotic‐guided endoscope; and finally, the Lucas‐Kanade optical flow algorithm was extensively tested in a visual servoing application, using a robotic endoscope.
Results
The combination of a Lucas‐Kanade tracking algorithm and a SURF‐based feature detection method gave the best performance in terms of accuracy and robustness of tracking, while preserving real‐time computation requirements. The optimal parameters consist of a window size of 51 × 51 pixels and an interframe motion threshold of 20 pixels. Feature tracking was successfully integrated into uncalibrated visual servoing or a robotic‐guided endoscope.
Conclusions
Robust feature tracking on a beating heart with endoscopic video can be achieved in real‐time and may facilitate robotically‐assisted, minimally invasive bypass surgery and conventional laparoscopic surgery. Copyright © 2011 John Wiley & Sons, Ltd.