Incorporating convolutions into transformers for textile fiber identification from fabric images

Simple, fast and effective fiber identification can help consumers purchase their desired apparel and help the industry conduct large-scale textile testing. This paper presents a transformer architecture incorporating convolutions to recognize fibers in textile surface images, which meets the above requirements. Firstly, a convolution operation is performed on textile images to pick up overlapping patches as tokens and the linear projections in transformer encoders are replaced by depth-wise separable convolutions to extract the fiber representations. Secondly, the multi-head cross-attention module enables each label embedding to be compared with features at each spatial location to locate and pool the corresponding fiber characteristics. Finally, a simplified asymmetric loss is introduced to further purify the extracted fiber features. Experiments demonstrate that the proposed approach provides a significant improvement in fiber identification accuracy over both state-of-the-art multi-label classification frameworks and fiber identification architectures.