Recognizing Unknown Disaster Scenes With Knowledge Graph-Based Zero-Shot Learning (KG-ZSL) Model

Unseen category prediction is a common challenge for real-world applications, especially for remote sensing (RS) imagery interpretation. Zero-shot learning (ZSL)--based scene classification methods have made significant progress recently, providing an effective solution for unseen scene recognition with semantic embeddings that link seen and unseen classes in the field of RS. However, existing ZSL methods mainly focus on semantic feature exploration, they failed to combine image features and semantic features effectively. To address the aforementioned challenges, we propose a novel knowledge graph-based zero-shot learning model that adeptly integrates both image and semantic features for disaster RS scene recognition. First, we construct an RS knowledge graph to generate semantic features of RS scenes, enhancing the reasoning ability from conventional RS scene categories to disaster RS scene categories. Second, we propose an Interactive Attention mechanism to integrate image and semantic features, focusing on the most informative regions. Finally, we introduce an RS domain adapter that enables the model to better adapt to remote sensing data, reproject common features into the remote sensing domain, and thus solve zero-shot remote sensing scene classification tasks. To demonstrate the effectiveness of our method, we construct a remote sensing disaster scene dataset, which contains 8700 high-quality disaster scenes. Extensive experiments show that our proposed method outperforms current state-of-the-art methods under zero-shot RS image scene classification settings.