Autofocusing With 3-D Tracking for Robot-Assisted Microsurgery

Robot-assisted microsurgery is gaining increasing attention in procedures requiring super-human dexterity and accuracy. In addition to challenges in micro-operation instrument design, defocus blur is a major obstacle in microsurgical robotics due to the shallow depth-of-field of microscopes and the dynamic nature of the scene. Different approaches to autofocusing have been explored, but thus far, they have mainly been limited to microscopes for cells and standard cameras. This article presents a real-time 3-D tracking framework for addressing unique defocusing challenges of surgical microscopes, including the dynamic instrument–tissue interaction, the severe reflection, and the paucity of surface features in an in vivo environment. In specific, novel order constraints are introduced for accurate defocus estimation. Based on defocus estimation, a nonblind virtual refocusing method is integrated into region-of-interest (ROI) tracking. Extensive offline and online experiments were performed, demonstrating the superior performance of defocus estimation and ROI tracking, and the effectiveness of the whole framework for autofocusing.