User Experience of a Conversational User Interface in a Museum

In this paper, we summarize the initial results of a field test with the ChiM (Chatbot in the Museum) system, a conversational user interface for the museum. The system contains a Natural Language Understanding (NLU) component that translates the user input into intentions and produces a multimodal (mainly spoken and textual) output. Museum visitors can use the system to freely ask questions about the exhibits in the exhibition. We conducted a field test with 140 participants in the Städel Museum, Frankfurt, and recorded over 4600 interactions between the participants and the system. After the test, participants gave their perceived feedback on the user experience (UX) and completed a custom system-specific questionnaire. We exploratively analyzed the feedback. The results show an overall medium UX for the system. We assume that the NLU component must be improved. Participants who rarely or never use audio guides rate the pragmatic quality (PQ) of the system significantly better than people who often or always use audio guides. People who rated the speech quality of the system as good also rated the attractiveness of the system significantly better than people who rated the speech quality as bad. In our future work, we will deepen the UX analysis and further put focus on recorded interaction data.