零(语言学)
适应(眼睛)
计算机科学
降噪
弹丸
蒸馏
考试(生物学)
语音识别
声学
人工智能
物理
材料科学
地质学
光学
化学
色谱法
语言学
古生物学
哲学
冶金
作者
Sunwoo Kim,Mrudula Athi,Guang Shi,Minje Kim,Trausti T. Kristjansson
出处
期刊:Journal of the Acoustical Society of America
[Acoustical Society of America]
日期:2024-02-01
卷期号:155 (2): 1353-1367
摘要
A personalization framework to adapt compact models to test time environments and improve their speech enhancement (SE) performance in noisy and reverberant conditions is proposed. The use-cases are when the end-user device encounters only one or a few speakers and noise types that tend to reoccur in the specific acoustic environment. Hence, a small personalized model that is sufficient to handle this focused subset of the original universal SE problem is postulated. The study addresses a major data shortage issue: although the goal is to learn from a specific user's speech signals and the test time environment, the target clean speech is unavailable for model training due to privacy-related concerns and technical difficulty of recording noise and reverberation-free voice signals. The proposed zero-shot personalization method uses no clean speech target. Instead, it employs the knowledge distillation framework, where the more advanced denoising results from an overly large teacher work as pseudo targets to train a small student model. Evaluation on various test time conditions suggests that the proposed personalization approach can significantly enhance the compact student model's test time performance. Personalized models outperform larger non-personalized baseline models, demonstrating that personalization achieves model compression with no loss in dereverberation and denoising performance.
科研通智能强力驱动
Strongly Powered by AbleSci AI