Readability, Fluency and Error identification when using Machine Translation and AI to translate medical research texts

With the improvements in the algorithms used to develop online machine translation tools and the growing relevance of AI there is a raising concern related to the quality of these tools’ outputs. Therefore, we sought to evaluate the readability of the results of machine translation and AI for the translation of medical research texts from English into European Portuguese. A total of 16 scientific papers were collected and two translations of each source text were obtained, using Google Translate and ChatGPT, (n=32). Several readability formulas and indexes were assessed, as well as text fluency, fidelity and linguistic errors’ occurrence. An analysis according to the selected translation tool revealed no significant differences between the results originated in Google Translate and ChatGPT, with regards to the quantitative readability analysis, meaning that their performance was similar despite the perceived fluency and fidelity issues. However, discourse errors were more common in the texts originated from ChatGPT, (p<0.001). These results are encouraging and support the use of machine translation tools and AI to translate medical research texts; however, human review and post-editing cannot be ruled out since the algorithms’ results largely depend on text complexity.