Defect classification for specular surfaces based on deflectometry and multi-modal fusion network

Automated defect inspection for specular surfaces is still a challenge in the manufacturing industry because of their specular reflection property. Deflectometry provides surface information based on the captured fringe patterns through the reflection of the specular surfaces and has been widely applied in defect detection for specular surfaces. Conventional methods combined deflectometry with machine learning approaches, but the hand-crafted features need to be defined for each specific task. Combined with the deep neural network, the input images are obtained from deflectometry, and the network completes the identification of the defects. Nevertheless, conventional deep-learning-based defect inspection methods approached the problem as a binary classification, or only certain obvious defects can be correctly classified. In this study, we generated and released, for the first time, to the best of our knowledge, the benchmark dataset named SpecularDefect9 with various defects for specular surfaces, and the classification accuracy of some kinds of defects may be low with only one kind of input image. To classify all kinds of defects accurately, the proposed method applied the light intensity contrast map combined with the original captured fringe pattern as the input of the network, and a fusion network was introduced to extract features from multi-modal inputs. Experimental results based on the released benchmark dataset verified the effectiveness and robustness of the proposed multi-modal defect classification method.