Leveraging Timbre Semantic Descriptors for Cross-Dataset Instrument Recognition

Timbre, a perceptual attribute of musical sound, plays a critical role in both instrument recognition and timbre perception research. However, these fields are rarely integrated, and as a result, their findings rarely benefit one another. This study investigates how integrating timbre semantic descriptors as an auxiliary task can enhance musical instrument recognition under challenging cross-dataset conditions via multi-task learning (MTL). A standard flat MTL framework is introduced, alongside an asymmetric MTL proposed to amplify the influence of timbre semantic descriptors on instrument recognition. Experiments are conducted on Western and Chinese instrument datasets to evaluate cross-cultural performance. The results demonstrate significant improvements on both datasets using the flat MTL, with further analysis revealing the varying contributions of individual timbre descriptors and demonstrating its robustness across different hyperparameter settings. The asymmetric MTL achieves additional performance gains on Western datasets but not on Chinese ones. A dedicated analysis is carried out to investigate this discrepancy, offering insights into potential dataset-specific or cultural challenges.