EfficientNet-Transformer for image captioning in Bahasa

The role of image captioning task that gives visual understanding of images enables semantic-based information retrieval. The research on image captioning always aims to produce better descriptions of images. There is only a little research found regarding image captioning in Bahasa Indonesia. All the studies use sequence-to-sequence (seq2seq) models alongside attention mechanisms. The model gives good results. However, there are crucial drawbacks. The seq2seq model gives poor performance when dealing with long sentences, while the attention mechanism consumes a lot of resources because it relies on Recurrent Neural Network (RNN). Inspired by the success of the Transformer architecture in machine translation, this research focuses on developing the image captioning model using Transformer architecture in Bahasa. The Transformer architecture accelerates the learning process since it only uses attention mechanisms without relying on RNN. Moreover, we use EfficientNet, one of the state-of-the-art architectures to extract features of the images. We use the MS COCO 2014 dataset with translated captions (in Bahasa). We did several experiments through hyperparameter tuning and selected the best model. The best model in our study gives a BLEU- {1,2,3,4} score of {77.42, 67.11, 60.52, 50.46}. The evaluation score indication and the inference result of the model, developed by EfficientNet-Transformer architecture, gives a very good, generated caption result.