Sequential Endoscopic-image 3D Reconstruction using Structured-light and Neural Signed Distance Field with Photometric Loss

The 3D shape measurement of internal organs is crucial for both diagnosis and surgery using endoscopy and colonoscopy. While various attempts have been made to develop 3D endoscopic systems, active one-shot scanning methods are particularly advantageous as they integrate seamlessly with existing systems. However, these methods suffer from issues such as noise and reduced accuracy due to unstable pattern detection. To address these challenges, we propose an approach based on neural signed distance field (neural SDF), incorporating a pattern reflection model. Our method, while utilizing structured light, eliminates the need for decoding by modeling structured light patterns as point light sources with high-frequency illumination characteristics. By employing differentiable rendering to minimize the difference between rendered and actual images, our approach enables the simultaneous estimation of object shape, surface reflection properties (including texture and shading), and camera pose. Furthermore, we extend our method to endoscopic Simultaneous Localization and Mapping (SLAM), where traditional feature-based approaches fail due to the lack of distinct shape and texture features in internal organs by sequentially optimizing camera pose through differentiable rendering for each frame, achieving robust wide-area shape reconstruction. Experimental validation using rendered images of a polygon model of an actural colon demonstrated the effectiveness of our approach in comparison to previous methods.Clinical relevance- Endoscopic shape measurement is relevant to cancer diagnosis, computer-assisted interventions, and making depth-annotation for machine learning training data.