计算机科学
人工智能
变压器
上下文图像分类
可转让性
计算
模式识别(心理学)
机器学习
计算机视觉
图像(数学)
物理
罗伊特
算法
量子力学
电压
作者
Leyuan Fang,Kehu Yang,Qiang Liu,Yi Yang,Jun Yue
出处
期刊:IEEE Transactions on Geoscience and Remote Sensing
[Institute of Electrical and Electronics Engineers]
日期:2023-01-01
卷期号:61: 1-13
标识
DOI:10.1109/tgrs.2023.3336283
摘要
Large-scale pre-trained models, such as vision transformers, have made significant progress in remote sensing (RS) scene classification tasks. For a new scene classification task, it is popular to fully fine-tune the pre-trained model parameters to avoid training from scratch. Although such an approach achieves satisfactory results, it will lead to heavy computation and storage burden, which limits the transferability of large pre-trained models to different RS scene classification tasks. To address this challenge, we propose a parameter-efficient tuning approach called as the Instance-Aware Visual Prompting (IVP), which is the first work to explore the prompting in the field of RS scene classification. The proposed IVP adaptively generates prompts based on the complex background and highly variable characteristics of RS images, and updates only a few parameters to transfer the pre-trained RS Transformer model to different scene classification tasks. Specifically, instead of adapting the entire model parameters, we introduce some instance-specific prompt vectors into the input space. Then, considering the significant variability in RS images, we introduce an instance-level prompt generation module to generate specific prompts for each RS image by aggregating contextual information from the input. Finally, these prompt vectors will calibrate the pre-trained features to encode instance-specific information. Extensive experiments on three RS scene classification datasets demonstrate the superiority of IVP over other fine-tuning methods. For example, when updating just 1.1% parameters, the Swin Transformer model achieves about 1.83% and 1.42% improvement compared to the full fine-tuning method on NWPU-19 and NWPU-28, respectively.
科研通智能强力驱动
Strongly Powered by AbleSci AI