Explicit Sparse Attention Network for Multimodal Named Entity Recognition

Multi-modal named entity recognition (MNER) is a multi-modal task aim to discover named entities in text with visual information. Existing MNER approaches model dense interactions between visual objects and textual words by designing co-attention mechanisms to achieve better accuracy. However, mapping interactions between each semantic unit (visual object and textual word) will force the model to calculate irrelevant information, which results in the model’s attention to be distracted. In this paper, to tackle the problem, we propose a novel model which concentrates the model’s attention by explicitly selecting the most relevant segments to predict entities. This method based on top-k selection can reduce the interference caused by irrelevant information and ultimately help the model to achieve better performance. Experimental results on benchmark dataset demonstrate the effectiveness of our MNER model.