Attitudes Toward the Use of Artificial Intelligence Chatbots for Mental Health Support: Artificial Intelligence in Mental Health Scale

Background/Objectives: Measuring attitudes toward the use of Artificial Intelligence (AI)-based chatbots for mental health support is highly important, especially as these technologies become more integrated into clinical and therapeutic settings. The aim of our study was to develop and validate a scale to measure attitudes toward the use of AI-based chatbots for mental health support, i.e., the Artificial Intelligence in Mental Health Scale (AIMHS). Methods: A multidisciplinary panel of experts assessed the content validity of items that were developed after an extensive literature review. To confirm face validity, we carried out cognitive interviews and calculated the item-level face validity index. Furthermore, we applied both exploratory and confirmatory factor analyses to verify the construct structure of the AIMHS. Moreover, we assessed measurement invariance across demographic subgroups, specifically examining differences by gender, age, and daily use of AI chatbots, social media platforms, and websites. Concurrent validity was evaluated using three instruments: the Artificial Intelligence Attitude Scale (AIAS-4), the Attitudes Towards Artificial Intelligence Scale (ATAI), and the Short Trust in Automation Scale (S-TIAS). Finally, reliability was tested through Cronbach’s alpha, Cohen’s kappa, and the intraclass correlation coefficient. Results: Exploratory and confirmatory factor analyses supported a two-factor model -technical personal advantages- that explained 81.28% of the variance. Moreover, the AIMHS demonstrated strong concurrent validity, evidenced by statistically significant correlations with AIAS-4, ATAI, and S-TIAS. Configural measurement invariance and metric invariance were supported by our findings. Cronbach’s alpha for the AIMHS was 0.798, and intraclass correlation coefficient was 0.938. Cohen’s kappa for the five items ranged from 0.760 to 0.848. Conclusions: The AIMHS is a psychometrically sound and user-friendly instrument for assessing attitudes toward the utilization of AI-based chatbots in mental health support.