S2F2AN: Spatial–Spectral Fusion Frequency Attention Network for Chinese Herbal Medicines Hyperspectral Image Segmentation

Chinese herbal medicine (CHM), as a treasure of the Chinese nation, is critically important for ensuring therapeutic efficacy through quality identification. Due to the subtle spatial feature differences among certain homologous CHM, existing identification methods based on red, green, blue (RGB) imaging often lack accuracy. In contrast, hyperspectral image (HSI) offers high spatial resolution and rich spectral information, effectively addressing this limitation. However, current HSI-based classification methods frequently fail to fully exploit spatial and spectral features, resulting in low accuracy when directly applied to the identification of CHM with subtle spectral differences. To achieve the differentiation of homologous CHM, we developed a proprietary hyperspectral imaging system to construct the first HSI dataset specifically for CHM quality assessment, featuring pixel-level annotations. To efficiently utilize the spatial and spectral information of HSI and enhance identification accuracy, we conceptualized feature extraction as a frequency filtering problem and designed a spatial-spectral fusion frequency attention network ($\text {S}^{2}\text {F}^{2}$AN). The network comprises a spatial-spectral frequency attention (SSFA) module, consisting of parallel continuous spatial frequency modules and spectral frequency attention modules, which perform multiscale adaptive filtering in the frequency domain to filter out redundant features and enhance the representation of the most discriminative spatial and spectral features, achieving spatial-spectral feature perception under a global receptive field. In addition, we proposed a cross-feature fusion (CFF) module that facilitates the mutual guidance of spatial and spectral features, ensuring the retention and fusion of key features. The experimental results indicate that on our self-constructed dataset, the average values of MIoU, MDice, and MPa reached 0.936, 0.967, and 0.974, respectively, surpassing existing methods.