SLResNet: Neural-Network-Based End-to-End Structure Light 3D Reconstruction for Endoscope

Three-dimensional endoscopes have been widely studied and developed for video-guided mini-invasive surgery to improve depth perception. Modern 3D endoscopes still cannot provide accurate depth readings. Moreover, the users may suffer from symptoms such as dizziness and nausea resulting from the vergence-accommodation conflict on stereo endoscopes. To resolve the aforementioned challenges, this work takes advantage of both structured light techniques and neural-network-based methods to reconstruct depth information on endoscopic images. The developed method includes SLResNet, a neural network model for end-to-end structure light pattern decoding, and a coordinate refinement algorithm. To focus on algorithm design, this work evaluated the algorithms on the projector-camera system. Using a metal gauge block as the targeted object, the maximum relative depth error is 0.396mm. This method can reconstruct at steepness up to 70 degrees stably. The errors in reconstructing the human upper jaw are less than 1mm in depth.