Critical Engagement with Generative AI: Development and Validation of a Scale

The use of large language model-based generative artificial intelligence (GenAI) in knowledge work differs fundamentally from traditional technologies, as these systems are probabilistic models trained on vast, diverse data. Their adoption by knowledge workers necessitates “skillful navigation”. Despite prior work on “effective” and “mindful” use, existing scales provide only a partial understanding of the nuances of GenAI interaction. To address this gap, we follow MacKenzie et al. (2011) and best practices in scale development to propose a multidimensional construct- Critical Engagement with GenAI. Drawing on the literature on critical thinking, problem-solving, and skepticism, and informed by focus group insights, we conceptualize this construct as reflective and refine the items into two sub-dimensions: Thoughtful Structuring of Input and Skeptical Review of Output, using content validity. Exploratory factor analysis supports this theoretical conceptualization, which we propose to further validate by using confirmatory factor analysis, providing a foundation for measuring skillful GenAI use.