Consistency between GAI’s and Teachers’ Rating and Corrective Feedback in English Writing Task

This study aims to explore the consistency between generative artificial intelligence (GAI) and teachers in assessing English writing tasks.A total of 15 university students participated in a threemonth teaching experiment focused on IELTS Writing Task 2. Two teachers and two generative AI systems (KIMI and ChatGPT 4.0) independently scored 120 essays and provided written feedback.By calculating the consistency between the AI and teacher scores, it was found that the internal consistency of the two AI systems was higher than that of the two teachers, while the consistency between the AI and teachers was moderate.In terms of score distribution, the AI systems exhibited a relatively concentrated trend, with scores generally higher than the students' actual English writing proficiency.Through coding and analysis of the feedback provided by teachers and AI on the students' essays, it was observed that the two AI systems consistently adhered to the assessment criteria of IELTS Writing Task 2 across all 120 essays.In contrast, the teachers occasionally displayed subjective tendencies and did not fully cover all four scoring dimensions of the task.Additionally, differences were noted in the manner in which teachers and AI delivered their feedback.The study concludes that, at present, teachers cannot fully rely on AI for essay assessment.However, collaboration between teachers and AI in evaluating writing tasks can enhance efficiency while effectively mitigating inconsistencies and subjectivity arising from human factors.