When ChatGPT can't Chat: The Quest for Naturalness

Abstract Significant advancements in voice chatbots have spurred interest into their role in second language learning (Conium, 2008), particularly their ability to assist in the development of learners' conversation skills in a target language. Many efforts have been made to explore AI's potential to act as conversation partners for language learners. Of central concern is its “naturalness” in such an endeavor (Yang et al., 2022). Despite the importance of gauging “naturalness” in researching human–AI interaction, we are yet to find a systematic way of doing so. In this report, drawing upon half‐a‐century of conversation analytic findings as well as 15 recordings of human–AI interaction, we propose a framework that features three principles of natural human interaction: emergence, indexicality, and recipient design. Our goal is to support empirical research aimed at improving “naturalness” of AI conversational partners with language learners.