Assessment of AI ethical reflection: the development and validation of the AI ethical reflection scale (AIERS) for university students

Abstract This study developed and validated the AI Ethical Reflection Scale (AIERS), a tool for measuring university students’ ethical reflection on AI in three dimensions: AI ethical awareness, AI critical evaluation, and AI for social good. This study used a sample of 730 university students, and confirmatory factor analysis supported the three-dimensional structure of the AIERS, which demonstrated good internal consistency and construct validity. Additionally, we found evidence of convergent and discriminant validity, as AI ethical reflection was positively yet modestly correlated with AI literacy, indicating that AI ethical reflection is a related yet distinct construct from AI literacy. The study also explored the differences in AI ethical reflection based on gender, academic discipline, and prior AI experience. The results revealed that female students exhibited higher AI ethical awareness than male students. Furthermore, frequent AI users reported significantly greater AI ethical reflection across all dimensions than those who used AI less frequently or had never used it. However, no significant differences were found between Science, Technology, Engineering, Mathematics (STEM) and non-STEM students. Collectively, these findings emphasize the importance of integrating ethical considerations and hands-on AI experiences from diverse perspectives of gender in AI education across disciplines to promote the ethical and responsible use of AI among university students.