Establishing the Criterion-related, Construct, and Content Validities of a Simulation-based Assessment of Inquiry Abilities

AbstractThe emphasis on scientific inquiry has increased the importance in developing the fundamental abilities to conduct scientific investigations and urged a need for valid assessments of students' inquiry abilities. We took advantage of the advanced technology to develop a simulation-based assessment of inquiry abilities (SAIA) that allowed students to generate scientific explanations and demonstrate their experimental abilities. This paper describes the validation of the assessment. Data were collected from 48 12th-grade students at a local high school who were categorized into three groups based on their program majors. Both quantitative and qualitative approaches were utilized to validate SAIA. The quantitative results showed that SAIA was aligned with a validated reasoning-skill test (criterion-related validity), discriminated variance among different groups (construct validity), and was highly suitable for examining inquiry abilities (content validity). Additionally, we utilized the think-aloud technique in order to identify the performances exhibited by students while they accomplished the SAIA tasks. The protocol analysis indicated that in general, students demonstrated the expected abilities in SAIA and that their SAIA scores accurately reflected their performance levels of inquiry abilities. The results suggested that SAIA was a valid assessment for evaluating the inquiry abilities of high school students. This study also provided systemic strategies for validating simulation-based assessments.Keywords: ValidationSimulation-based assessmentInquiry ability AcknowledgmentsThis study was based upon work supported by the National Science Council of Taiwan under NSC 100-2511-S-003-042-MY3 and the Aim for the Top University Project at the National Taiwan Normal University.Notes1. Howell (Citation2007) suggested a simple formula, , for estimating the sample size. In this study, we assumed that the effect size (d) is equal to .50, so for power = .80, α = .05, and a two-tailed designed the number of students per sample group (n) would be equal to 15.37. Therefore, we needed 16 students per sample group for a total of 48 students.