Face, Construct and Criterion Validity, and Test-Retest Reliability, of the Adult Rejection Sensitivity Questionnaire

Abstract: This research sought to test the face, construct and criterion validity, and test-retest reliability of the Adult Rejection Sensitivity Questionnaire (ARSQ). In Study 1, participants ( n = 45) completed the ARSQ and questions assessing scale item relevancy, clarity, difficulty, and sensitivity. In Study 2, participants ( n = 513) completed the ARSQ and demographic questions. In Study 3, participants ( n = 244) completed the ARSQ and returned 2 weeks later to complete the ARSQ and measures of depression, anxiety, and self-silencing behavior. Study 1 provided strong support for face validity with all items deemed relevant, clear, easy to answer, and neither distressing nor judgmental. Study 2 provided adequate support for the factor structure of the ARSQ (single-factor model and two-factor model) but suggested modifications could be made to improve scale validity. Study 3 provided further support for an adequate (but not good) factor structure, and evidence for criterion validity established through medium-large effect size correlations with depression, anxiety, and self-silencing behavior. However, the 2-week scale stability appeared poor ( r = .45) in a subsample of participants. Overall, the ARSQ showed sufficient validity to recommend its continued use, but we recommend further tests of scale reliability and potential modifications to increase construct validity.