Research on Tibetan-Chinese Machine Translation Method Based on Graphic Multimodal Fusion Alignment

This article explores a Tibetan-Chinese machine translation model based on multimodal alignment of images and texts, using the Resnet50 model for feature extraction of images, the TIP-LAS Tibetan Segmentation Lexical Annotation System tool for lexical annotation and segmentation of Tibetan texts, and the Stuttering tool for lexical annotation and segmentation of Chinese texts. Experiments on the same graphic Tibetan-Chinese dataset and CMXT2022 Tibetan-Chinese dataset show that the Tibetan-Chinese machine translation model based on graphic multimodal alignment is higher than the plain text Tibetan-Chinese machine translation translation-model based on Transformer by 2.09 BLEU points and 2.33 BLEU points, respectively.