An empirical study of ChatGPT use in engineering education: Prompting and performance

This study investigates how specific prompting behaviors influence the academic performance of engineering students using ChatGPT. A total of 128 fourth-year students across four engineering programs participated in weekly sessions involving rotating task types: case analysis, engineering design, multi-step problem-solving, and experimental data analysis. A Python-based interface connected students to ChatGPT-4o via the API, logging all interactions and implementing stratified randomization of AI access across the 16-week semester. Written assignments were used for both grading and analyzing AI content integration. Ten metrics were designed to capture student behaviors—eight focusing on prompting (AI Query Count, Query Depth and Structure, Query Efficiency, Prompt Refinement Depth, Response Utility, Response Complexity, Response Reliance, and AI-Driven Problem-Solving) and two assessing writing quality (Structural Complexity Score and Content Richness and Information Density Score). Inter-metric analysis examined how these behaviors related to one another and to academic performance. Results showed that AI Query Efficiency, which reflects how effectively students crafted clear, well-structured prompts, and AI-Driven Problem-Solving, which captures how strategically they integrated AI output into their reasoning, were the strongest predictors of academic success. These findings were supported by Mann–Whitney U tests comparing AI and non-AI groups, as well as Spearman correlations, Random Forest regressors, partial dependence plots, principal component analysis, and mixed-effects modeling, and remained significant even after accounting for students' cumulative GPA. The results suggest that prompting strategy plays a meaningful role in shaping how effectively students use AI in engineering education.