Assessing Scientific Practices Using Machine Learning Methods: Development of Automated Computer Scoring Models for Written Evolutionary Explanations

Although multiple-choice assessment formats are commonly utilized throughout the educational hierarchy, they are only capable of measuring a small subset of important disciplinary competencies and practices.Consequently, science educators require openresponse format assessments that can validly measure more advanced skills and performances (e.g., producing written scientific explanations).However, open-response format assessments are not practical in many educational contexts because of the high cost of scoring, the delayed feedback to test-takers, and the lack of scoring consistency among human graders.In this study, the efficacy of automated computer scoring (ACS) of written explanations is examined relative to human scoring.This study aims to build ACS models using machine-learning methods in order to detect a suite of scientific and naive ideas in written scientific explanations, and to explore approaches for optimizing these ACS models.This study develops and evaluates nine machine-learning models to detect the scientific concepts of variation, heritability, competition, limited resources, differential/survival, and non-adaptive ideas and the naive concepts of needs/goals, use/disuse, and adapt/acclimation.In addition, it examines the effects of three machinelearning parameters (i.e., n-gram selection, stop words, and misclassified data) on the performance of the ACS models.In order to test the efficacy of the ACS models, a corpus of 10,270 written evolutionary explanations--in response to a variety of items differing in Many people have contributed their time, talents, and other resources to this dissertation work.First and foremost, I would like to express my heartfelt gratitude to my advisor, Dr. Ross Nehm, who always supports, encourages, and challenges me to do better research and be a better researcher.The knowledge, expertise and advice that I have received from him has formed the core of my academic scholarship.Your patience, support and understanding of my circumstances have brought me to where I am