TeDFL: Toward Text-Driven Few-Shot Learning for Forest Tree Species Classification With Airborne Hyperspectral Images

Tree species classification is fundamental for forest inventory and sustainable silviculture. Airborne hyperspectral images can capture subtle spectral differences between tree species through dense spectral channels, aiding the discrimination of similar species. However, in multi-species field scenarios, mixed pixels and environmental interference exacerbate inter-class spectral similarity and intra-class spectral variation, making it difficult to accurately distinguish tree species solely through visual representation. To this end, a text-driven few-shot learning (TeDFL) paradigm is developed that can leverage linguistic prior knowledge to guide the generation of discriminative visual features. The aim of the TeDFL method is to train a highly flexible and robust model that can adapt to classifying diverse yet similar tree species using just a few examples. Our TeDFL framework contains two primary encoders: a visual encoder and a text encoder. The former exploits a ghost attention network (GANet) to explore spatial-spectral visual features without excessive parameters, where GANet consists of ghost and normalization-based attention blocks. The latter leverages the contrastive language-image pretraining model to incorporate additional rich semantic priors, enriching the visual feature space. Furthermore, we design cross-modal alignment (CMA) and cross-modal integration (CMI) strategies to bridge the modal discrepancy between visual and textual representations. The CMA strategy aims to facilitate a visual-text contrastive objective by contrasting hyperspectral images with class-specific text prompts, while the CMI strategy adaptively fuses multimodal support data to obtain more reliable prototypes. Experiments on four realistic forest scenarios confirm the feasibility and effectiveness of the TeDFL method compared with other state-of-the-art methods. Moreover, the TeDFL method could serve as a standard visual-language backbone network for few-shot tree species classification research. The source code will be made publicly available at https://github.com/HlEvag/TeDFL.git.