From ‘Nativelike’ to Practical: Comparing Human and AI Reformulation as Written Feedback

This study investigated the feasibility of using artificial intelligence (AI)-generated reformulations as written feedback in second language (L2) writing contexts. A comparative analysis was conducted to assess the degree of lexical resemblance between reformulated texts by human individuals and those by AI. Following Cohen's definition of reformulation, seven first language (L1) English as a foreign language (EFL) university instructors and three types of AI chatbots − ChatGPT-3.5, Claude2 and Bing Chat Creative Mode − reformulated picture description narratives written by five L1 Japanese EFL university students. Utilizing corpus analysis tools, the study examined text lengths, lexical sophistication, lexical diversity and incorporation of new vocabulary and multiword units. The findings revealed that among all the reformulator types, human-generated reformulations most closely resembled the original drafts in text length and vocabulary usage. Furthermore, among the AI systems, Bing Chat Creative Mode was found to be the most comparable to human reformulators in these aspects. In contrast, ChatGPT-3.5 and Claude2, showcased more advanced vocabulary and greater lexical diversity. The study concludes with pedagogical implications.