HFENet: A lightweight hand‐crafted feature enhanced CNN for ceramic tile surface defect detection

International Journal of Intelligent SystemsEarly View RESEARCH ARTICLE HFENet: A lightweight hand-crafted feature enhanced CNN for ceramic tile surface defect detection Fangfang Lu, Fangfang Lu Shanghai University of Electric Power, Shanghai, ChinaSearch for more papers by this authorZhihao Zhang, Zhihao Zhang Shanghai University of Electric Power, Shanghai, ChinaSearch for more papers by this authorLingling Guo, Lingling Guo College of Chemical Engineering, Zhejiang University of Technology, Hangzhou, ChinaSearch for more papers by this authorJingjing Chen, Corresponding Author Jingjing Chen joyjchan@gmail.com orcid.org/0000-0003-1737-3420 Fudan University, Handan Road, Shanghai, China Correspondence Jingjing Chen, Fudan University, Handan Road, Shanghai, 200433 China. Email: joyjchan@gmail.comSearch for more papers by this authorYihan Zhu, Yihan Zhu College of Chemical Engineering, Zhejiang University of Technology, Hangzhou, ChinaSearch for more papers by this authorKe Yan, Ke Yan National University of Singapore, Singapore, SingaporeSearch for more papers by this authorXiaokang Zhou, Xiaokang Zhou Shiga University, Shiga, JapanSearch for more papers by this author Fangfang Lu, Fangfang Lu Shanghai University of Electric Power, Shanghai, ChinaSearch for more papers by this authorZhihao Zhang, Zhihao Zhang Shanghai University of Electric Power, Shanghai, ChinaSearch for more papers by this authorLingling Guo, Lingling Guo College of Chemical Engineering, Zhejiang University of Technology, Hangzhou, ChinaSearch for more papers by this authorJingjing Chen, Corresponding Author Jingjing Chen joyjchan@gmail.com orcid.org/0000-0003-1737-3420 Fudan University, Handan Road, Shanghai, China Correspondence Jingjing Chen, Fudan University, Handan Road, Shanghai, 200433 China. Email: joyjchan@gmail.comSearch for more papers by this authorYihan Zhu, Yihan Zhu College of Chemical Engineering, Zhejiang University of Technology, Hangzhou, ChinaSearch for more papers by this authorKe Yan, Ke Yan National University of Singapore, Singapore, SingaporeSearch for more papers by this authorXiaokang Zhou, Xiaokang Zhou Shiga University, Shiga, JapanSearch for more papers by this author First published: 26 May 2022 https://doi.org/10.1002/int.22935Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Inkjet printing technology can make tiles with very rich and realistic patterns, so it is widely adopted in the ceramic industry. However, the frequent nozzle blockage and inconsistent inkjet volume by inkjet printing devices, usually leads to defects such as stayguy and color blocks in the tile surface. Especially, the stayguy in complex pattern is difficult to identify by naked eyes due to it is easily covered by complex patterns and becomes invisible, this brings great challenge to tile quality inspection. Nowadays, the machine learning is employed to address the issues. The existing machine learning methods based on hand-crafted features are capable of stayguy detection of the tiles with a simple pattern, but not applicable for complex patterns due to the interference of pattern in feature extraction. The emerging deep-learning-based methods have the potential to be applied for stayguy detection with complex patterns, but cannot achieve real-time detection due to high complexity. In this paper, a lightweight hand-crafted feature enhanced convolutional neural network (named HFENet) is proposed for rapid defect detection of tile surface. First, we perform data enhancement on the original image by global histogram equalization and image addition. Second, for the special shape of stayguy which is usually vertical, we embed the extended vertical edge detection operator (Prewitt) as convolution kernel into HFENet to extract the hand-crafted vertical edge features of the test image and eliminate the interference of complex pattern in the feature extraction. Third, the 5 × 1 asymmetric convolution kernel with a dilation rate of 2 is used to improve the utilization of convolution kernel and reduce the complexity of the model. Fourth, to reach the real-time requirements, a memory access cost-aware design is proposed, which can orchestrate the number of shallow convolution layers and deep convolution layers in feature extraction. The experiments were performed on the ceramic tile image data set captured by high-resolution industrial cameras in ceramic tile production line. Experimental results show that the HFENet outperforms the state-of-the-art semantic segmentation networks (i.e., UNet, FCN-8s, SegNet, DeepLabV3+, etc.) and lightweight networks (i.e., ShuffleNet, MobileNet, and SqueezeNet). All the code and data are available at a GitHub repository (https://github.com/RobotvisionLab/HFENet). Early ViewOnline Version of Record before inclusion in an issue RelatedInformation