Dual-stream reinforcement network for few-shot image segmentation

Deep learning has achieved successful results in semantic segmentation of images. However, semantic segmentation of images still faces challenges in real-world applications. On the one hand, due to the repeated stacking of convolution layer and pooling layer, each feature information is localized, eliminating long-term dependence and content deviation. On the other hand, it requires large number of annotated dataset for training, and can not classify unknown categories. To address the above issues, a dual-stream reinforcement network (DRNet) is proposed for few-shot image semantic segmentation in the paper. The dual-stream branches are designed, including a prototype enhancement branch and a query-guided branch. The former branch is designed to compute query vectors based on feature similarity. The latter branch is presented to extract the expression of target features and feature consistency information between images by utilizing additional memory units. Moreover, a matrix of feature correlations is generated by a graph attention mechanism between support vectors and query vectors, which allows query information to be propagated to support vectors. Finally, additional memory units are used to mine the representation of target features. In this way, images with consistent features and information about common features can be effectively fused. The experimental results show that the proposed model DRNet can achieve state-of-the-art results compared to other models. Notably, the mean-IoU scores of our model are 53.34% and 62.03% for PASCAL-5i at 1-shot and 5-shot settings, respectively, which are more competitive than other methods.