地标
计算机科学
人工智能
深度学习
语音识别
一般化
特征(语言学)
字错误率
阅读(过程)
模式识别(心理学)
可视化
计算机视觉
特征提取
图像(数学)
网格
图像处理
性格(数学)
作者
Srujith Rao Ambati,Sayma Akther
标识
DOI:10.1109/icdsns65743.2025.11168793
摘要
Lip-reading bridges computer vision and speech processing by recognizing spoken words from visual lip movements alone. This study presents a streamlined framework combining facial landmark detection, image enhancement, and deep spatiotemporal modeling. We use MTCNN to detect and align lip regions, enhanced by Real-ESRGAN for higher resolution and finer detail. The enhanced images feed into a 3D CNN with timedistributed layers and bidirectional LSTM, trained using CTC loss for effective spatial-temporal feature learning and alignmentfree transcription. Evaluated on the GRID corpus, our model achieves a character error rate (CER) of 2.3% on seen speakers and 5.2% on unseen speakers. Overall, it delivers state-of-the-art performance with a 5.2% CER and 95.8% accuracy, improving CER by 18.8% over LipNet. Notably, for unseen speakers, it reduces CER from LipNet's 9.4% to 5.2%, a 44.7% relative decrease, showcasing superior generalization and robustness. These results highlight that combining super-resolution with deep temporal modeling substantially enhances visual speech recognition accuracy and reliability.
科研通智能强力驱动
Strongly Powered by AbleSci AI