Understanding the User Experience of Customer Service Chatbots: What Can We Learn from Customer Satisfaction Surveys?

Understanding and improving user experience is key to strengthening uptake and realizing the potential of chatbots for customer service. In this paper, we investigate customer satisfaction surveys as a source of insight into such user experience. A total of 5,687 customer satisfaction reports on users' interactions with a customer service chatbot, and the corresponding chatbot interactions, are analyzed. The findings demonstrate that customer satisfaction reports are closely associated with the degree to which the problems motivating users' chatbot interactions are resolved. Furthermore, the findings show substantial variation in the performance of different chatbot intents in terms of customer satisfaction and problem resolution. This implies that user experience varies substantially depending on the problems motivating users to interact with the chatbot. Finally, we identify key characteristics of the intents associated with particularly high or low customer experience, suggesting paths towards efficient improvement of chatbot user experience. Based on the findings, we point to key implications for theory and practice and suggest directions for future research.