Enhancing Critical Writing Through AI Feedback: A Randomized Control Study

This study investigates the effectiveness of artificial intelligence-generated content (AIGC) systems on undergraduate writing development through a randomized controlled trial with 259 Chinese students. Despite promising applications of AI in educational settings, empirical evidence regarding its comparative effectiveness in writing instruction remains limited. Using a four-week intervention comparing Qwen-powered AI feedback to traditional instructor feedback, we employed difference-in-differences (DiD) analysis and structural equation modeling to examine how technology acceptance factors influence writing outcomes. Results demonstrated significant improvements in the AIGC intervention group compared to controls (β = 0.149, p < 0.001), with particularly strong effects on organization (β = 0.311, p < 0.001) and content development (β = 0.191, p < 0.001). Path analysis revealed that perceived usefulness fully mediated the relationship between perceived ease of use and attitudes toward the system (β = 0.326, p < 0.001), with attitudes strongly predicting behavioral engagement (β = 0.431, p < 0.001). Contrary to traditional technology acceptance models, perceived ease of use showed no direct effect on attitudes, suggesting that students prioritize functional benefits over interface simplicity in educational technology contexts. These findings contribute to an expanded technology acceptance model for educational settings while providing evidence-based guidelines for implementing AI writing assistants in higher education.