Learning With Mobile Augmented Reality- and Automatic Speech Recognition-Based Materials for English Listening and Speaking Skills: Effectiveness and Perceptions of Non-English Major English as a Foreign Language Students

Due to the outbreak of COVID 19, an online bilingual curriculum was conducted via Google Meet. The learning material was developed and implemented by using a smartphone application, STEMUP, based on augmented reality (AR) and automatic speech recognition (ASR) technologies. This study investigated the oral performance and perceptions of learning with STEMUP of ninety non-English major students from several colleges at a technical university in Taiwan. Data were collected from pre- and post-tests and a questionnaire survey. Results indicated that students significantly improved their oral performance and recorded their positive perceptions. Students’ oral performance significantly depended on their English proficiency. Their perceptions were not significant related to their English proficiency, gender, or college. Instant feedback and evaluation provided by ASR technology and online Google text-to-speech service both embedded in STEMUP helped students notice, modify and improve their listening and speaking skills. They were satisfied with the bilingual curriculum, which helped them increase understanding about content knowledge by the teacher’s explanation in Chinese, and improve English listening and speaking skills by learning with STEMUP. This study is a good start in creating an interactive and communicative learning environment where translanguaging is effectively integrated with innovative technologies.