Algorithmic Self-Deception: How AI-Generated Feedback Skews Learners' Self-Reflection

This study investigated how artificial intelligence (AI)-generated feedback from large language models like ChatGPT affected undergraduate students’ self-assessment accuracy and reflective depth. Conducted with 180 students from Makerere University (Uganda) and the University of Cape Town (South Africa), the research explored the impact of AI feedback compared to human and no feedback, focusing on self-assessment accuracy, reflection quality, and cross-cultural differences. The objectives were to: (i) assess the impact of AI feedback on self-assessment accuracy; (ii) measure its effect on reflection depth and quality; and (iii) compare responses between Makerere and UCT students, considering cultural and contextual factors. Using a mixed-methods experimental design, participants were randomly assigned to receive AI feedback, human feedback, or no feedback on essays. Quantitative data analyzed with ANOVA and t-tests showed that AI feedback improved essay scores (mean 82.5%) significantly over human feedback (80.1%) and no feedback (73.2%). However, AI feedback led to greater overconfidence, evidenced by higher calibration errors (t(58) = 3.28, p = .002), and produced reflections of lower quality compared to human feedback (F(2,177) = 26.4, p < .001). Qualitative analysis revealed that Makerere students tended to trust AI feedback more and exhibited stronger overconfidence than their UCT counterparts. Findings highlighted an “algorithmic self-deception” effect, where AI-generated feedback’s vague positivity inflated learners’ self-perceptions and diminished critical reflection. The study recommended incorporating AI literacy training, developing hybrid human–AI feedback models, and designing culturally sensitive AI tools to foster deeper metacognitive engagement. These strategies were vital to harness AI’s educational potential while addressing its limitations across diverse cultural settings.