Evaluating the Quality of AI-Generated Digital Educational Resources for University Teaching and Learning

With the proliferation of artificial intelligence in education, AI-generated digital educational resources are increasingly being employed as supplements for university teaching and learning. However, this raises concerns about the quality of the content produced. To conduct a comprehensive quality assessment, this paper presents an evaluation index system for AI-generated digital educational resources by combining the Delphi method and the Analytic Hierarchy Process. The initial quality indicators across the dimensions of content, expression, and user and technical aspects are identified through a systematic literature review of the recent research. Then, the Delphi method is utilized to modify the quality indicators according to experts’ opinions through two rounds of questionnaire surveys. Subsequently, the weight coefficients of the quality indicators are calculated using the Analytic Hierarchy Process. Finally, a quality indicator system for evaluating AI-generated digital educational resources is developed, which comprises four dimensions and twenty indicators. The findings reveal that content characteristics are of critical importance in assessing the quality of AI-generated educational resources, followed by expression characteristics as the second most significant factor, with user and technical characteristics also being recognized. Among the second-level indicators, “authenticity”, “accuracy”, “legitimacy”, and “relevance” are accorded greater importance relative to other indicators. The proposed system equips relevant stakeholders with a framework for selecting high-quality AIGDERs and steering AI tools in line with educational standards. Finally, some implications are provided to support the selection of high-quality AI-generated resources and guidance on aligning these resources with educational standards.