Distance Transformer for Image Captioning

The architecture based on Transformer represents the state of the art in sequence modeling tasks. However, their applicability to multi-modal contexts, such as image captioning, has not been fully studied. In this paper. We propose a new image captioning model based on Transformer. It reconstructs the self-attention in Transformer by introducing distance dependent operator in attention modeling between different words, thus solving the problem of wrong attention in the application of Transformer in image captioning field. Our method is evaluated on the MSCOCO dataset and achieves state-of-the-art performance, improving the best result in terms of BLEU-4 from 38.4 to 39.2.