Examining generative artificial intelligence (GenAI) in assessing students’ responses to socio-scientific issues in physics

Despite the potential benefits of generative artificial intelligence (GenAI), most research in science education has focused on its advantages, with limited evidence on how it can assist humans in specific roles, such as assessment. This study provides empirical evidence of GenAI as a potentially useful tool with topic-dependent consistency, comparable to science teacher educators or human intelligence (human experts). Specifically, it examines the capacity of different GenAIs to assess students’ responses in the form of socio-scientific issues (SSI) reasoning or argumentation, a prominent focus in science education. A case study approach was used to analyze data from 22 first-year physics education students at a private university in Indonesia, with SSI instruments adapted from the Victorian Curriculum and Assessment Authority covering thermodynamics, mechanics, and electricity. Both GenAIs and human intelligence assessed students’ responses. The findings revealed that ChatGPT, Gemini, Copilot and human intelligence were consistent in scoring SSI argumentation for thermodynamic topic, whereas they showed inconsistencies for mechanics and electricity. While GenAIs and human intelligence captured similar keywords, they sometimes categorised them differently across SSI aspects. The study concludes by discussing the implications of these findings, offering insights for educators on leveraging GenAI as an evaluation tool.