Statistical Consequences of Attribute Misspecification in the Rule Space Method

The present study investigates the statistical consequences of attribute misspecification in the rule space method for cognitively diagnostic measurement. The two types of attribute misspecifications examined in the present study are exclusion of an essential attribute (which affects problem-solving performance) and inclusion of a superfluous attribute (which does not). Results of a simulation study show that exclusion of an essential attribute tends to lead to underestimation of examinees’ mastery probabilities for the remaining attributes, whereas inclusion of a superfluous attribute generally leads to overestimation of attribute mastery probabilities for the other attributes. In addition, order relations among attributes induced by superset/subset relationships affect the biases in the estimated attribute mastery probabilities in systematic ways. These results underscore the importance of correct attribute specification in cognitively diagnostic assessment and delineate some specific effects of using incorrect attribute sets.