The Concept of Validity in Psychological Assessment

The concept of validity is central to psychological assessment, providing the theoretical and methodological principles for the development and use of measurement instruments. A consensus has emerged that validity does not reside in the measuring instrument per se, but rather in the inferences drawn from the scores. However, the concept of validity is complex and its intricacies continue to be debated. In this chapter, we aim to represent the diversity of perspectives on the concept of validity and to translate the implications of these perspectives for psychological assessment. There has been a movement away from the historical emphasis on types of validity (e.g., content, criterion, construct) and from the view of reliability as distinct from, but related to, validity. These terms have been reconceptualized as different forms of evidence gathered through the process of validation to support the claim to validity. This “unitary” approach to validity has gained traction, though some texts still refer to different types of validity. What counts as evidence in support of validity depends on the basic assumptions about what is being measured (substantive theory) and about how it should be measured (measurement theory). Substantive theory addresses the nature of the phenomena under consideration (i.e., realist and constructivist perspectives) and measurement theory addresses the principles and procedures to quantify psychological phenomena and to gather evidence of validity. Measurement theory enjoys considerable consensus, but questions regarding substantive theory remain unsettled. Quantitative and qualitative measurement share commonalities in the conception of validity but rely on different validation procedures. As emphasized in the Standards for testing endorsed by educational and psychological professional associations, decisions based on tests are consequential for people’s lives, warranting consideration of all available evidence, including issues of bias and fairness in the interpretation and the use of scores. Yet when considering the implications of test scores for decision-making, it is hard to escape basic questions about the nature of the phenomenon being measured by a particular method. Gaps between substantive theory and validation procedures, including use of metrics that don’t adequately represent the target phenomenon, reduce the usefulness of conclusions drawn. Different instruments purporting to measure the same phenomenon may capture different aspects of that phenomenon or may apply in different contexts (hence, low agreement is not fully explained by measurement error). Since clinical assessments often include batteries of tests that get at different psychological phenomena relevant to the issues at hand, a more complete conception of validity in psychological assessment would reach beyond the current emphasis on validation of single test scores.