Deep learning-based inpainting of high dynamic range fringe pattern for high-speed 3D measurement of industrial metal parts

To guarantee the manufacturing quality of industrial metal parts, it is important for three-dimensional measuring their shapes on production line. When using digital fringe projection profilometry to measure the metal parts, that is, high dynamic range objects, the issue of local over-exposure arises. This over-exposure causes the loss of fringe pattern and phase information, resulting in a significant impact on the accuracy of three-dimensional measurement. To address this issue, an encoder-decoder network guided by a reflective prior is proposed. This network aims to inpaint the problematic fringe patterns caused by highly reflective surfaces. The objective is to convert these patterns into ideal fringe patterns with uniform gray levels. The proposed encoder-decoder network consists of an encoder network with a dual path guided by the reflective prior and a decoder network with partial convolution. By utilizing the residual module and transformer, a novel encoder module is constructed for extracting global feature information. Meanwhile, a reflective-prior guidance module is proposed and incorporated into the coding network to assist in estimating highly reflective areas. By embedding the partial convolution, a decoder network is designed for fusing the shallow semantic features. By incorporating multiple decoder modules, the decoder network is built to distinguish between pixels in the high reflection area and the non-high reflection area during the upsampling process. The proposed fringe inpainting method is used on the actual metal part dataset with high reflectivity. The experimental results indicate that the proposed method can handle various types of modulated fringe patterns, including those with different metal parts, fringe frequencies, and overexposure. Additionally, it can effectively eliminate the highly reflective area and restore missing fringe phase information. Consequently, it improves the accuracy of subsequent 3D reconstruction.