Vision Module for Automatic Tracking on Bedside Intelligent Scope‐Holding Surgical Robot System

ABSTRACT Background This study aims to develop an active following technology of the mirror‐holding arm of a bedside intelligent surgical robot that enables real‐time automatic tracking of surgical instruments. Methods The semantic segmentation network was adaptively enhanced, and new datasets were collected with surgical clamps as the target, incorporating scenarios commonly encountered during surgical procedures. In addition, the position of the clamp was determined, and the range and mode of motion of the robot arm were set. Based on the above premise, the control rate was designed and verified using an endoscopic surgery simulation platform. Results The average time for the collaborative arm to reach the centre of the field of view was 2.51 s, with a variance of 0.32. Conclusions The results of this study validate the effectiveness of replacing manual endoscope holding with a robotic arm, which has great potential to facilitate more efficient endoscopic surgical procedures.