Can AI replace Teachers? Comparing AI and Human Teachers’ ratings and feedback on students’ essays

Abstract This study aimed to evaluate the consistency between human and AI-generated ratings on ESL student essays and to compare the manner and focus of their feedback. Using a standardized rubric, three human raters and three AI tools assessed essays based on content, organization, language, and mechanics. Findings revealed that while the average ratings from both groups were generally comparable, the inter-rater consistency between humans and AI was low, indicating that the two sets of raters did not closely align in how they assigned scores. Despite this, statistical analysis using the Wilcoxon signed-rank test showed no significant difference in overall ratings, suggesting that both groups arrived at similar general evaluations despite differing in scoring patterns. In terms of feedback, human raters typically provided direct and corrective remarks focused on grammar and structure. In contrast, AI tools offered more varied and structured comments that addressed broader aspects of writing, such as clarity, organization, and idea development, often balancing praise with critique. The study concludes that while AI tools like ChatGPT, CiCi, and Bing show potential in supporting writing assessment, they should be used to complement—not replace—human evaluation. Human oversight remains essential to ensure accuracy, contextual understanding, and pedagogical appropriateness in student assessment. Continued training for teachers on how to effectively use and interpret AI-assisted feedback is recommended to maximize its educational value.