Deep‐Learning‐Based Real‐Time Visual Detection Method for Robotic Cell Microinjection System With High Accuracy and Efficiency

The advancement of the life sciences has led to an increased demand for biological cell microinjection techniques. A number of algorithms have been developed for the purpose of detecting cells, and employed in automated cell microinjection systems. However, due to the significant difference in scale between the microinjector tip size (~10 µm) and the cell size (~100 µm), conventional cell detection algorithms struggle to accurately detect both the microinjector tip and the cell simultaneously. Additionally, both the cells and the microinjector are transparent, making it difficult to distinguish them from the culture medium under the light. This leads to a lack of real‐time coordinate feedback for the cell and microinjector tip, increasing the risk of damaging the cell during puncture. To this end, this paper proposes a deep‐learning based real‐time visual detection method for robotic cell microinjection system with high accuracy and efficiency. The core of the method is the target detection algorithm based on improved YOLOv8n (denoted as BI‐YOLO). The deformable convolution is added to the backbone of algorithm to adapt to different shapes and sizes of cells and microinjector tips. And bidirectional feature pyramid network (BiFPN) which adds adaptive feature fusion is used as the neck part of the algorithm to achieve higher accuracy and precision for cell and microinjector tip detection using smaller computational resources. Zebrafish embryo injection experiments are conducted using the designed robotic micromanipulation system to verify the effectiveness of the proposed visual detection method. The experimental results show that the BI‐YOLO algorithm achieves an average intersection over union (IoU) accuracy of 99.5% for cells and 97.8% for microinjector tips, with just 1.96 million parameters and 7.2 giga floating point operations per second (GFLOPs) of computation. Its overall performance surpasses other popular deep learning algorithms. The success rates for cell and microinjector tip detection are 100% and 94.7%, respectively, while the detection speed is 118.8 frame rate per second (FPS). The visual detection method enables real‐time detection of positions of cell and microinjector tip and provide visual feedback. The method will be promising in the application of robotic batch cell microinjection with high efficiency.