Reverse Scored Items, Should We Use It on Psychology Scale or Not?

Although the use unfavorable item on the psychological scale was suggested by numerous authors, however, some studies found poor-quality psychometric property on such items. This study aimed to replicate those studies with emphasis on factorial and criterion validity of unfavorable items. Research data was obtained from the measurement of achievement motivation among high-school students (N=2986) using Achievement Motivation Scale. Analysis results from testing measurement model by exploratory factor analysis approach suggest that three factors model was fitted with the data. However, one of these factors contain all unfavorable items. This finding indicated that the emerging factors from analysis were caused by the similarity of method rather than indicator content. Correlation between items/factors and criterion show no criterion validity differences between favorable and unfavorable items. Similar results were also shown from correlation between three factors and the criterion. This study concludes that using unfavorable items on the scale tend to interrupt factorial validity but not criterion validity.