Differentiating translational English from original English using large language models

Abstract Translated language often carries subtle linguistic markers that set it apart from text originally written in the target language. This study investigates the potential of powerful AI language models to automatically identify these differences, specifically focusing on distinguishing translated texts from original texts. More specifically, this study utilized transformer-based large language models to differentiate translational English and original English. FLOB (Freiburg–LOB Corpus of British English) was selected as the original English corpus, and COCE (Corpus of Chinese-English) was selected as the translational English corpus. Two models were tested: Bidirectional Encoder Representations from Transformers (BERT) and Robustly Optimized BERT Pretraining Approach (RoBERTa). The factors affecting classification results are investigated through SHAP analysis and analysis of text types that have significantly different error rates. Results show that both models achieved excellent performance, with F1-scores of .864 for BERT and .998 for RoBERTa. The text types miscellaneous, general fiction, skills trade, and hobbies, and humor exhibit significantly higher error rates. Reportage, review, and science exhibit significantly lower error rates. Through SHAP analysis, we find that higher error rates may be attributed to simpler sentences in these three text types and the shared characteristics of translational texts, such as the tendencies for simplification and explicitation. Conversely, lower error rates were associated with text types that did not share these characteristics. In summary, transformer-based large language models show great potential for the automatic classification and analysis of translational sentences.