CrossHypergraph: Consistent High-Order Semantic Network for Few-Shot Image Classification

Few-shot classification is a challenging task that recognizes novel classes by learning from few training instances. Metric-based models are currently the most effective solutions for few-shot classification. In these models, patch feature distances between query instances and support classes are calculated to achieve classification. However, it is difficult for patch-based methods to mine semantic information of support and query instances, leading to inaccurate feature similarity measures. To address these problems, we propose to construct CrossHypergraph based on hypergraph modeling. Specifically, we first align the local prototype vertices of support and query instances to model consistent hypergraph structures. Then a vertex-hyperedge-vertex-based interactive feature updating mechanism is designed to generate CrossHypergraph representation with consistent high-order semantic information for support and query instances. Based on the CrossHypergraph, we propose a consistent high-order semantic network, in which the high-order semantic-based weighted metric strategy is designed to achieve accurate classification. The proposed method is evaluated on general, fine-grained, and cross-domain few-shot benchmarks, including miniImageNet, tieredImageNet, CIFAR-FS, FC100, and miniImageNet $\rightarrow$ CUB datasets. Experimental results show that our CrossHypergraph-based few-shot classifier generates consistent high-order semantic features, and achieves state-of-the-art performance on both 1-shot and 5-shot tasks.