Large Language Models for Psychological Assessment: A Comprehensive Overview

Large language models (LLMs) are extraordinary tools demonstrating potential to improve our understanding of psychological characteristics. They provide an unprecedented opportunity to supplement self-report in psychology research and practice with scalable behavioral assessment. However, they also pose unique risks and challenges. This article serves as an overview and guide for psychological scientists to evaluate LLMs for psychological assessment. In Section I, we briefly review the development of transformer-based LLMs and discuss their advances in natural language processing. In Section II, we describe the experimental design process including techniques for language data collection, audio processing and transcription, text preprocessing, and model selection, as well as analytic matters such as model output, model evaluation, hyperparameter tuning, model visualization, and topic modeling. At each stage, we describe options, important decisions, and resources for further in-depth learning, while providing examples from different areas of psychology. In Section III, we discuss important broader ethical and implementation issues and future directions for researchers using this methodology. The reader will develop an understanding of essential ideas and an ability to navigate the process of using LLMs for psychological assessment.