Perceived conversational ability of task-based chatbots – Which conversational elements influence the success of text-based dialogues?

The use of text-based chatbots offering individual support to customers has increased steadily in recent years. However, thus far, research has focused on comparing text-based chatbots with either each other or with humans, whilst the investigation of task-based dialogues has been scarce. This paper aims to identify the characteristics of dialogues – that is, conversational elements – that lead to a successful task-based conversation. For this purpose, the chatbot, KIM, by MAGGI Kochstudio was used. It was designed to help customers find a recipe tailored to their individual needs. In order to investigate which conversational elements contribute to successful communication between the user and the chatbot KIM, a usability study collecting 123 unstructured dialogues and a scenario-based experiment using four dialogues with 627 respondents was conducted. The quantitative analysis demonstrates that task completion is characterized by a higher perception of the chatbot’s conversational ability and user satisfaction. The chatbot should propose correct recipe suggestions following a short dialogue, without the user needing to provide too much input. Based on these findings, we recommended equipping the skillset of task-based chatbots with elements that will complement their assistive qualities – for example, improved use of standard phrases, and reactions to similar domains and non-requests. Gender-specific differences in task completion should be considered.