The many faces of scientific inquiry: Effectively measuring what students do and not only what they say

Abstract Science education frameworks in the United States have moved strongly in recent years to incorporate more dimensions of learning, including measuring student use of scientific practices employed during scientific inquiry. For instance, the Next Generation Science Standards and related multidimensional frameworks adopted or adapted recently by more than 30 United States include numerous complex science performance skills required of students. This article considers whether valid and reliable evidence can be obtained in online performance tasks to yield an estimate of both student inquiry practices and of the ability of students to explain their understanding of scientific concepts. A data set from a Virtual Performance Assessment (VPA) task, There's a New Frog in Town , is examined. Delivered through an online system, the VPA task engages students in guided inquiry through problem solving, modeling, and exploration. The VPAs are designed to produce evidence on more than one latent trait in the respondent performance. Results of the case study reported here indicated that maps of student proficiency in scientific inquiry were possible to generate from the VPA data set, using measurement models. Addition of process data through a new hybrid measurement model, mIRT‐Bayes, improved reliability of results. Results indicated overall that virtual performance tasks may be helpful for science assessment, especially if assessment time is short and a goal is to increase the validity and quality of performance measures with authentic and engaging virtual activities.