What is the impact of a multi‐modal pedagogical conversational AI system on parents' concerns about technology use by young children?

Abstract Previous research showed that the parents acknowledged the technology's benefits for their young children's learning, however, they are still worried about the extended screen time, lack of physical activity and lack of social interactions. To address these concerns, we developed Kid Space to enable pedagogically appropriate technology use for children in early childhood education by combining various sensing technologies with a multi‐modal conversational artificial intelligence system that can interact with children, understand individual progress and provide personalised learning experiences. To understand the impact of Kid Space on the parents' initial concerns about technology use by their young children, we conducted a multi‐method user study: (1) a quasi‐experimental design and (2) formative research method using an exploratory case study with a set of children and their parents experiencing Kid Space in their homes. The results show that after experiencing Kid Space with their children, the parents felt significantly less concerned about screen time, social interactions and physical activity and reported positive perceptions towards pedagogical value of Kid Space. Detailed analysis on the multi‐modal data quantitatively and qualitatively validated why Kid Space alleviated these concerns. Future research is needed to validate long‐term educational value of Kid Space and generate insights for improvement for next iterations. Practitioner notes What is already known about this topic Play‐based learning is critical for young children's education, but digital games create major concerns around extended screen time, lack of physical activity and lack of social interactions. Blending digital and physical spaces could support pedagogically appropriate technology use for young children. Towards this end, there are some exemplary studies in the state–of‐the‐art reporting positive educational outcomes as an effect of utilising such spaces. However, none of these studies supported children's most natural mode of communication in their interactions with the systems—speaking. Pedagogical conversational agents (PCAs) are promising, but they are tricky when it comes to young children's speech because of unique technical challenges resulted from how children use language and communicate with digital systems. What this paper adds To our best knowledge, Kid Space is one of the earliest implementations of a PCA with a multi‐modal artificial intelligence (AI) system utilising physical and digital learning manipulatives for maths learning with a focus on early childhood education. The key contributions of this paper are (1) the design and development of an end‐to‐end multi‐modal system enabling Wizard‐of‐Oz experimentation for initial evaluations with users, (2) the creation of a multi‐modal, in‐the‐wild labelled dataset with children–agent, children–parent and children–physical/digital space interactions enabling advancements for AI components for later evaluations with users and (3) the generation of rich insights from an initial research study on user perceptions and engagement as well as actionable findings to improve Kid Space experiences for next iterations and inform key design features for similar systems. Implications for practice and/or policy The results of the study implied a set of areas for improvement—or design features—for Kid Space and other similar pedagogical conversational systems developed for children's home usages: (1) easier setup and usage with optimised setup size addressing diverse space limitations at homes, (2) minimised latency between Oscar (the conversational pedagogical agent) and child interactions (eg, adding multimodal dialogue system to reduce the need for a human wizard), (3) more advanced personalisation, social (including more verbal interactions) and pedagogical skills for Oscar with increased contextual awareness (eg, sending children's engagement), (4) scalability and higher visual quality of content with diverse games and learning outcomes, (5) parental control features over Kid Space platform and Oscar (eg, time limit, content, etc.) and (6) accessibility features (eg, captions turned on for multilingual children) and support for neurodiversity.