Prevalence and Psychometric Implications of Careless Responses in an Online Student Survey

Surveys are widely used data collection tools in empirical studies of human behaviour. Self-reporting plays a central role in exploring various psychological processes that are integral to human behaviour and learning, such as motivations or emotions. However, respondents can sometimes be the source of measurement error in survey research. In this context, we investigated the phenomenon of ‘careless responding’, in which respondents fail to read or adequately attend to the constructs measured by survey items, which can result in data that deviate from participants’ true levels of the constructs being measured. Specifically, 405 undergraduate students, predominantly female (81.7%) with an average age of 21, were asked to complete the HEXACO Personality Inventory online. The inventory included several control items: two instructed response items, two reverse response items, and a self-report item to determine the prevalence of careless responders. Results indicated that the reverse items flagged a higher percentage of participants as careless responders compared to the instructed response items. However, the consistency of the instructed response items was much better. Only a small number of participants were flagged by the self-report item. Females exhibited greater carefulness and bonus points had no significant impact on carelessness. Subsequent analyses included inter-factor correlations, reliability statistics, descriptive statistics, and exploratory structural equation modelling results for both careful and careless responder groups, with the results worsening for the careless groups. These statistics were followed by implications that are discussed according to the literature on survey methodology.