Applied AI in Assessing Design Education: A Comparison of Design Progress Evaluation Between AI Dashboard and Conventional Human Evaluators

Abstract The process of design in education is generally broad, which makes it difficult to measure. It is human-centric in nature, thus assessment is often qualitative and involves comparing students’ performance or grading. The dynamic nature of design processes also takes a lot of time from educators to fully understand the progress made by student design teams in order to provide critical and essential feedback. As such, time becomes a valuable and important resource that educators need to ration to keep the pace of the class while ensuring students meet their learning objectives and milestones. This paper showcases a method of visualizing the design process of students through a dashboard using divergent and convergent thinking as metrics across individual, team, and cohort levels, with the use of Natural Language Processing (NLP) models. Such a dashboard allows educators a quantitative means to receive “real-time” feedback on the performance made in a large cohort of students and enables some optimization of the time spent assessing their progress. An evaluation of this approach compares the difference between the visualization of results produced by an AI-enabled dashboard and conventional human evaluators, showcasing similarities and contrasts. This paper ends with a discussion about the viability of the AI tool, presenting the current limitations and future potential.