Segformer-Mobilenetv3 fusion coordinate attention mechanism: A lite portrait segmentation network

Aiming at the problem of low accuracy of portrait segmentation, we proposes a lite mobile image segmentation algorithm, SegFormer MoibleNetv3(SMN), based on SegFormer and fusion coordinate attention mechanism MoblieNetv3 to achieve portrait segmentation. The improved MobileNetv3 is used as the backbone network for feature extraction, which can effectively reduce the size of the model and facilitate the migration and training of the model. SMN network structure can effectively learn portrait features, so as to separate portrait from background. The network structure can be selected to take portrait as input and output the corresponding image mask. Experiments were conducted on Matting, EG1800, and P3M-10K public datasets, The results show that the SMN algorithm outperforms the current mainstream lite portrait segmentation methods in terms of Mean Intersection over Union (MIoU), accuracy (Acc), model consistency (Kappa) and segmentation coefficient (Dice). MIoU index can reach 97.61% (Matting), 94.31%(EG1800), 95.64%(P3M-10K), accuracy (Acc) can reach 98.22%(Matting), 97.27%(EG1800), 97.97% (P3M-10K). The accuracy has been significantly improved, and the target person can be clearly separated from the background. The final model size is only 3.1M, which is smaller than SegFormer-B0 11.1M.The model in this paper can be applied to mobile phone and other mobile devices. Each image can reach more than 30 fps when processed on the test dataset on the experimental platform.