The role and impact of ChatGPT in educational practices: insights from an Australian higher education case study

Abstract Artificial intelligence (AI) tools, notably ChatGPT, are increasingly recognised for their transformative potential in higher education. This study employs a detailed case study approach complemented by a survey, delving into ChatGPT’s impact on pedagogical practices, student engagement, and academic performance. It involved 74 undergraduate and postgraduate students enrolled in data analytics courses in Australia. The quantitative analysis highlights ChatGPT’s role in providing personalised and on-demand support, which is highly valued among users for its flexibility and responsiveness, meeting a critical demand in educational settings. Notably, the study identifies a medium effect size ( $$\eta ^2 = 0.173$$ η 2 = 0.173 ) in perceived benefits, indicating that ChatGPT accounts for approximately 17.3% of the variance in improved academic outcomes. However, challenges such as ChatGPT’s limited understanding of complex queries and the lack of human interactions are primary concerns, with a medium effect size ( $$\eta ^2 = 0.289$$ η 2 = 0.289 ) suggesting significant areas for improvement. Furthermore, statistical analyses reveal a clear relationship between the frequency of ChatGPT usage and the perception of its benefits, underscoring the transformative potential for users who have integrated it into their academic practices. Despite these challenges, the differential impact on users versus non-users highlights the potential for ChatGPT to foster more engaging and effective educational practices. The findings advocate for targeted strategies to epitomise ChatGPT’s integration into educational settings, emphasising the need for ongoing research and the development of comprehensive guidelines to navigate its complexities and maximise its educational benefits.