Understanding the Empathetic Reactivity of Conversational Agents: Measure Development and Validation

With the advancement of artificial intelligence, conversational agents are now capable of displaying intelligent and emotionally empathetic responses, which are essential for the continued use of AI-based agents. However, there is a scarcity of formal measures that can comprehensively evaluate how well they react to the user's emotional needs. The objective of this research is to develop and validate a set of new measures, collectively called the agent empathic reactivity index (AERI), an adaptation of the interpersonal reactivity index (IRI) developed for the human-human relationship evaluation to the human-agent interaction context. By rigorously following the measure development procedures suggested by prior research, four dimensions of AERI measures of empathic concern, perspective-taking, fantasy, and personal distress were developed. Multiple pilot tests and surveys involving various conversational agents were conducted to validate the four AERI measures. The study results show that the new measures have strong psychometric properties and nomological validity.