Modality Imbalance? Dynamic Multi-Modal Knowledge Distillation in Automatic Alzheimer's Disease Recognition

Alzheimer's disease (AD), as the most prevalent form of dementia, necessitates early identification and treatment for the critical enhancement of patients' quality of life. Recent studies strive to explore advanced machine learning approaches with multiple information cues, such as speech and text, to automatically and precisely detect this disease from conversations. However, these multi-modality-based approaches often suffer from a modality-imbalance challenge that leads to performance degradation. That is, the multi-modal model performs worse than the best mono-modal model, although the former contains more information. To address this issue, we propose a Dynamic Multi-Modal Knowledge Distillation (DMMKD) approach, which dynamically identify the dominant modality and the weak modality, and opt to conduct an inter(cross)-modal or intra-modal knowledge distillation. The core idea is to balance the individual learning speed in the multi-modal learning process by boosting the weak modality with the dominant modality. To evaluate the effectiveness of the introduced DMMKD algorithm, we conducted extensive experiments on two publicly available and widely used AD datasets, i. e., ADReSSo and ADReSS-M. Compared to the multi-modal approaches without dealing with the modality imbalance issue, the introduced DMMKD indicates substantial performance improvements by 15.4% and 10.9% in terms of relative accuracy on the ADReSSo and ADReSS-M datasets, respectively. Moreover, when compared to the state-of-the-art models for automatic AD detection, the DMMKD achieves the best performance of 91.5% and 87.0% accuracies on the two datasets, respectively.