Exploring Global and Local Hierarchies: Dual Classifier With Mutual Distillation for Hierarchical Text Classification

As a pivotal variant of multi-label classification, hierarchical text classification (HTC) faces unique challenges due to its intricate taxonomic hierarchy. Recent state-of-the-art approaches improve performance by considering both global hierarchy covering all labels and local hierarchy indicating substructure of sample-specific ground-truth labels. However, they often over-condense hierarchical information into one or several tokens, which may cause the loss of useful knowledge. Accordingly, we propose a dual classifier model with global and local hierarchies (DCGL). It adopts prompt tuning-based BERT as the backbone, where global hierarchy is integrated into the soft prompt template. And this resulting classifier branch is termed global pipeline. To mitigate information loss caused by hierarchy condensation, we introduce a parallel local hierarchy-aware classifier pipeline. This local pipeline acquires label-level classification features through text propagation on the label hierarchy and aligns these features with oracle label representations of local hierarchy via graph contrastive learning, which serve as a novel strategy for local hierarchy incorporation. Thereby, DCGL obtains more granular and targeted features and captures local hierarchy information such as label co-occurrence and local structure. Moreover, since global and local pipelines capture distinct yet complementary information, we further apply mutual knowledge distillation to bridge the gap between their output logits and facilitate mutual learning. And to better control the distillation degree, we design a dynamic temperature negatively correlated with label confidence. Comprehensive experiments demonstrate that our DCGL outperforms several representative HTC methods.