Evaluating the Translation of Figurative Language: A Comparative Study of ChatGPT and Human Translators

Abstract Recently, Generative Pretrained Transformer (GPT) has demonstrated significant advancements in a variety of language-related tasks, including machine translation. However, many studies that evaluated the performance of ChatGPT in translation tasks focused on general texts. Consequently, the primary objective of this study was to assess how well GPT-3, can translate figurative language content from Arabic into English and vice versa and compare its performance with that of human translators. To achieve this objective, the study selected some passages from Gharkas (2003) focusing on different topics which included figurative language. These passages underwent translation by both a professional human translator and GPT-3. The study evaluated translation performance of GPT-3 against human translation using qualitative criteria. The criteria included accuracy, fluency, cohesion, and coherence, and translating figurative language. The findings of the analysis of these passages revealed that GPT-3 generated translations that were generally comprehensible but fell short in capturing figurative language in comparison to human translation. The results indicated that GPT-3 rendered similar number of sentences if the ST was English. However, when the ST was Arabic, GPT-3 reduced the length of long Arabic sentences and divided them to be shorter.