Towards Balanced Active Learning for Multimodal Classification

Training multimodal networks requires a vast amount of data due to their\nlarger parameter space compared to unimodal networks. Active learning is a\nwidely used technique for reducing data annotation costs by selecting only\nthose samples that could contribute to improving model performance. However,\ncurrent active learning strategies are mostly designed for unimodal tasks, and\nwhen applied to multimodal data, they often result in biased sample selection\nfrom the dominant modality. This unfairness hinders balanced multimodal\nlearning, which is crucial for achieving optimal performance. To address this\nissue, we propose three guidelines for designing a more balanced multimodal\nactive learning strategy. Following these guidelines, a novel approach is\nproposed to achieve more fair data selection by modulating the gradient\nembedding with the dominance degree among modalities. Our studies demonstrate\nthat the proposed method achieves more balanced multimodal learning by avoiding\ngreedy sample selection from the dominant modality. Our approach outperforms\nexisting active learning strategies on a variety of multimodal classification\ntasks. Overall, our work highlights the importance of balancing sample\nselection in multimodal active learning and provides a practical solution for\nachieving more balanced active learning for multimodal classification.\n