Masked Vision Transformer for Fast Hyperspectral Image Classification

Vision Transformer (ViT) has been thoroughly explored in hyperspectral image (HSI) classification (HIC). Nevertheless, current ViT-based approaches still acquire discriminative features, resulting in relatively limited generalization capabilities when confronted with the challenges posed by high intraclass variances and interclass similarities commonly observed in HSI data. Moreover, most of these methods fail to adequately emphasize the significance of the central pixel in HIC. To address the aforementioned challenges, we introduce a masked ViT (MViT) for HIC. First and foremost, MViT endeavors for the first time to introduce the masking operations in supervised models to learn more robust patterned features instead of distinguishable features, thereby bestowing it with outstanding generalization performance. Secondly, during the training phase of MViT, when conducting the random masking operations on the embedded features, we deliberately retain the embedding corresponding to the central pixel to guarantee the effectiveness of the model and emphasize the importance of the central pixel in HIC. Finally, MViT will deactivate the masking operations during the testing phase and utilize all the embedded features to accomplish the classification task, thereby enabling the model to fully exploit its recognition capabilities. On top of that, MViT is an extremely lightweight model, and by introducing the masking operations during the training phase, its training speed becomes unprecedentedly rapid. Experiments conducted on four publicly accessible datasets demonstrate that MViT can consistently achieve excellent or even the optimal classification results in comparison with the most advanced methods. The source code was powered by Jupyter and released at https://github.com/swiftest/MViT.